[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:03 dist-test-slave-5l3k krb5kdc[6380](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:03 dist-test-slave-5l3k krb5kdc[6380](info): set up 2 sockets
Aug 06 06:46:03 dist-test-slave-5l3k krb5kdc[6380](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 06:46:04.246764 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:04.367205 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44771
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:44771
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:04.448021 6393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:04.448208 6393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:04.448230 6393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:04.449255 6393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:04.449287 6393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:04.449301 6393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:04.449313 6393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:04.449326 6393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:04.450655 6393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:44771
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:04.450843 6393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:04.451021 6393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:04.455770 6399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.455902 6393 server_base.cc:1034] running on GCE node
W20260806 06:46:04.455818 6401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.456001 6398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.456291 6393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:04.456609 6393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:04.457798 6393 hybrid_clock.cc:648] HybridClock initialized: now 1785998764457787 us; error 47 us; skew 500 ppm
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:04.460309 6393 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:04.460757 6393 webserver.cc:460] Webserver started at http://127.6.57.126:38655/ using document root <none> and password file <none>
I20260806 06:46:04.460973 6393 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:04.461036 6393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:04.461194 6393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:04.462193 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "6dc001171c3f4578b1b4e1fec0e0dff5"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "1d78743bb9111949207b7f2f23461eb4"
server_key_iv: "69d214e21007cb1a9bbdbd98de7ad601"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.462608 6393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "6dc001171c3f4578b1b4e1fec0e0dff5"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "1d78743bb9111949207b7f2f23461eb4"
server_key_iv: "69d214e21007cb1a9bbdbd98de7ad601"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.463923 6393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:04.464604 6409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.464771 6393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:04.464871 6393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "6dc001171c3f4578b1b4e1fec0e0dff5"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "1d78743bb9111949207b7f2f23461eb4"
server_key_iv: "69d214e21007cb1a9bbdbd98de7ad601"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.464954 6393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:04.477154 6393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:04.479475 6393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:04.479630 6393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:04.483741 6393 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:44771
I20260806 06:46:04.483791 6463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:44771 every 8 connection(s)
I20260806 06:46:04.484164 6393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:04.485059 6465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:04.487418 6465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: Bootstrap starting.
I20260806 06:46:04.488005 6465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:04.488286 6465 log.cc:824] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:04.488950 6465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: No bootstrap required, opened a new log
I20260806 06:46:04.490464 6465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } }
I20260806 06:46:04.490684 6465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:04.490710 6465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc001171c3f4578b1b4e1fec0e0dff5, State: Initialized, Role: FOLLOWER
I20260806 06:46:04.490896 6465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } }
I20260806 06:46:04.490958 6465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:04.491014 6465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:04.491063 6465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:04.491688 6465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } }
I20260806 06:46:04.491751 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 6393
I20260806 06:46:04.491812 6465 leader_election.cc:302] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dc001171c3f4578b1b4e1fec0e0dff5; no voters:
I20260806 06:46:04.491865 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:04.491981 6465 leader_election.cc:288] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:04.492009 6468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:04.492146 6468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [term 1 LEADER]: Becoming Leader. State: Replica: 6dc001171c3f4578b1b4e1fec0e0dff5, State: Running, Role: LEADER
I20260806 06:46:04.492250 6465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:04.492236 6468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } }
I20260806 06:46:04.492559 6468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dc001171c3f4578b1b4e1fec0e0dff5. Latest consensus state: current_term: 1 leader_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } } }
I20260806 06:46:04.492633 6468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:04.492708 6373 external_mini_cluster.cc:1428] Setting key 37525e11933b33630a515505096c349e
I20260806 06:46:04.492811 6469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc001171c3f4578b1b4e1fec0e0dff5" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44771 } } }
I20260806 06:46:04.492882 6469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5 [sys.catalog]: This master's current role is: LEADER
W20260806 06:46:04.494400 6484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:04.494467 6484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:04.494578 6473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:04.494750 6473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:04.496497 6473 catalog_manager.cc:1347] Generated new cluster ID: 1274f0f8a2854172a74dc43794d6de59
I20260806 06:46:04.496551 6473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:04.505733 6473 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:04.506313 6473 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:04.509444 6473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dc001171c3f4578b1b4e1fec0e0dff5: Generated new TSK 0
I20260806 06:46:04.509609 6473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:04.511325 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.493818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.6.57.126:44771)
0806 06:46:04.494162 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:04.494172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:04.494584 (+ 412us) server_negotiation.cc:407] Connection header received
0806 06:46:04.497009 (+ 2425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:04.497017 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:04.497070 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:04.497225 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:04.499070 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.500972 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.502394 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.503047 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.503172 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.507400 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:04.507404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:04.507405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:04.507416 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:04.509297 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.509881 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.509885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.509885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.509978 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.510244 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.510245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.510245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.510473 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:04.510690 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:04.510964 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 06:46:04.511109 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:04.539462 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:04.621033 6491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:04.621270 6491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:04.621307 6491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:04.622473 6491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:04.622602 6491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:04.622650 6491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:04.624181 6491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:04.624436 6491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:04.624666 6491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:04.629709 6497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.629987 6491 server_base.cc:1034] running on GCE node
W20260806 06:46:04.629703 6499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.629710 6496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.630302 6491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:04.630585 6491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:04.631765 6491 hybrid_clock.cc:648] HybridClock initialized: now 1785998764631752 us; error 26 us; skew 500 ppm
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:04.634298 6491 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:04.634872 6491 webserver.cc:460] Webserver started at http://127.6.57.65:40847/ using document root <none> and password file <none>
I20260806 06:46:04.635169 6491 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:04.635228 6491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:04.635354 6491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:04.636307 6491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "65339c7994dd4819911b6335574af373"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "051b4756117bf9d0691d68be73159a5f"
server_key_iv: "b65a36dfe984b49c16c805b7e02efaaf"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.636653 6491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "65339c7994dd4819911b6335574af373"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "051b4756117bf9d0691d68be73159a5f"
server_key_iv: "b65a36dfe984b49c16c805b7e02efaaf"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.638072 6491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:04.638931 6507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.639132 6491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:04.639225 6491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "65339c7994dd4819911b6335574af373"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "051b4756117bf9d0691d68be73159a5f"
server_key_iv: "b65a36dfe984b49c16c805b7e02efaaf"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.639305 6491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:04.650213 6491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:04.652177 6491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:04.652340 6491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:04.652563 6491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:04.652879 6491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:04.652935 6491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.652969 6491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:04.652997 6491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.659390 6491 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:33645
I20260806 06:46:04.659449 6626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:33645 every 8 connection(s)
I20260806 06:46:04.659744 6491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:04.664426 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 6491
I20260806 06:46:04.664501 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:04.664666 6373 external_mini_cluster.cc:1428] Setting key 2f316d7c3b51d3fa43374294593fb075
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:04.669761 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.660376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44771 (local address 127.6.57.65:42249)
0806 06:46:04.660598 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:04.660660 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:04.661202 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:04.661461 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:04.661466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:04.662041 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:04.662251 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.662256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.663036 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.663037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.663838 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.663840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.664257 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.664258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.664402 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.665007 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:04.665015 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:04.666474 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:04.668448 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.668455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.668457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.668735 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.668942 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.668942 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.668943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.669038 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.669310 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:04.669314 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:04.669420 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 06:46:04.669498 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:46:04.669606 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260806 06:46:04.669811 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.660425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:42249 (local address 127.6.57.126:44771)
0806 06:46:04.660473 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:04.660475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:04.660754 (+ 279us) server_negotiation.cc:407] Connection header received
0806 06:46:04.661311 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:04.661313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:04.661326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:04.661393 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:04.662365 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.662940 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.663931 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.664160 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.664245 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.666594 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:04.666596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:04.666596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:04.666603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:04.668356 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.668805 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.668806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.668807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.668873 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.669100 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.669101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.669102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.669247 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:04.669315 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:04.669544 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 06:46:04.669667 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:04.670606 6627 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44771
I20260806 06:46:04.670701 6627 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:04.670892 6627 heartbeater.cc:511] Master 127.6.57.126:44771 requested a full tablet report, sending...
I20260806 06:46:04.671464 6427 ts_manager.cc:194] Registered new tserver with Master: 65339c7994dd4819911b6335574af373 (127.6.57.65:33645)
I20260806 06:46:04.672005 6427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:42249
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:04.693951 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:04.776038 6632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:04.776260 6632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:04.776297 6632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:04.777432 6632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:04.777496 6632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:04.777552 6632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:04.779126 6632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:04.779387 6632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:04.779656 6632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:04.784670 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.784646 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.784657 6641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.784755 6632 server_base.cc:1034] running on GCE node
I20260806 06:46:04.785006 6632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:04.785283 6632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:04.786458 6632 hybrid_clock.cc:648] HybridClock initialized: now 1785998764786448 us; error 39 us; skew 500 ppm
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:04.788959 6632 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:04.789404 6632 webserver.cc:460] Webserver started at http://127.6.57.66:41309/ using document root <none> and password file <none>
I20260806 06:46:04.789584 6632 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:04.789624 6632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:04.789712 6632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:04.790756 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "720b0ae1c80041919df1d3ae3cc8bd90"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "a7e940b3b2e55b945cbdb215e342e6f7"
server_key_iv: "4891ec5b38b2a34b5542b0f4878c5982"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.791074 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "720b0ae1c80041919df1d3ae3cc8bd90"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "a7e940b3b2e55b945cbdb215e342e6f7"
server_key_iv: "4891ec5b38b2a34b5542b0f4878c5982"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.792443 6632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:46:04.793272 6649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.793444 6632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:04.793532 6632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "720b0ae1c80041919df1d3ae3cc8bd90"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "a7e940b3b2e55b945cbdb215e342e6f7"
server_key_iv: "4891ec5b38b2a34b5542b0f4878c5982"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.793583 6632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:04.805150 6632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:04.807214 6632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:04.807322 6632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:04.807495 6632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:04.807797 6632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:04.807829 6632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.807848 6632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:04.807891 6632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.814317 6632 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:44213
I20260806 06:46:04.814384 6767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:44213 every 8 connection(s)
I20260806 06:46:04.814769 6632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:04.819124 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 6632
I20260806 06:46:04.819231 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:04.819397 6373 external_mini_cluster.cc:1428] Setting key 8dc36a9998cf71be7697983fc968ccdd
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:04.824530 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.815471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44771 (local address 127.6.57.66:34743)
0806 06:46:04.815663 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:04.815696 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:04.816194 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:04.816481 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:04.816485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:04.817045 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:04.817228 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.817233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.817956 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.817957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.818853 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.818856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.819256 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.819258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.819391 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.819928 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:04.819936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:04.821230 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:04.823221 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.823227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.823228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.823442 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.823663 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.823663 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.823664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.823757 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.824038 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:04.824042 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:04.824163 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:46:04.824246 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:46:04.824363 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260806 06:46:04.824543 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.815514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:34743 (local address 127.6.57.126:44771)
0806 06:46:04.815560 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:04.815562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:04.815777 (+ 215us) server_negotiation.cc:407] Connection header received
0806 06:46:04.816302 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:04.816303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:04.816316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:04.816380 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:04.817340 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.817859 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.818967 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.819175 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.819250 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.821326 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:04.821328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:04.821328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:04.821336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:04.823133 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.823524 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.823525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.823525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.823593 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.823823 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.823824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.823824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.823972 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:04.824049 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:04.824298 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 06:46:04.824399 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:04.825207 6768 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44771
I20260806 06:46:04.825307 6768 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:04.825500 6768 heartbeater.cc:511] Master 127.6.57.126:44771 requested a full tablet report, sending...
I20260806 06:46:04.825910 6427 ts_manager.cc:194] Registered new tserver with Master: 720b0ae1c80041919df1d3ae3cc8bd90 (127.6.57.66:44213)
I20260806 06:46:04.826395 6427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:34743
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:04.845793 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:04.928953 6773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:04.929167 6773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:04.929204 6773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:04.930321 6773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:04.930387 6773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:04.930431 6773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:04.931941 6773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44771
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:04.932214 6773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:04.932435 6773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:46:04.937299 6773 server_base.cc:1034] running on GCE node
W20260806 06:46:04.937264 6778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.937294 6781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:04.937275 6779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:04.937759 6773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:04.938035 6773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:04.939183 6773 hybrid_clock.cc:648] HybridClock initialized: now 1785998764939171 us; error 32 us; skew 500 ppm
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:04.941650 6773 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:04.942126 6773 webserver.cc:460] Webserver started at http://127.6.57.67:35431/ using document root <none> and password file <none>
I20260806 06:46:04.942358 6773 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:04.942444 6773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:04.942625 6773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:04.943662 6773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "33eb160e7da344e99e3d382758ec59f2"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "d232888f1ff16f4ffe8d4dbdb0bf4829"
server_key_iv: "210c56f67b052be55f0c26ef0a5f226c"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.944038 6773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "33eb160e7da344e99e3d382758ec59f2"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "d232888f1ff16f4ffe8d4dbdb0bf4829"
server_key_iv: "210c56f67b052be55f0c26ef0a5f226c"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.945351 6773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:04.946022 6788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.946187 6773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:04.946292 6773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "33eb160e7da344e99e3d382758ec59f2"
format_stamp: "Formatted at 2026-08-06 06:46:04 on dist-test-slave-5l3k"
server_key: "d232888f1ff16f4ffe8d4dbdb0bf4829"
server_key_iv: "210c56f67b052be55f0c26ef0a5f226c"
server_key_version: "encryptionkey@0"
I20260806 06:46:04.946363 6773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:04.957652 6773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:04.959604 6773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:04.959760 6773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:04.959964 6773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:04.960287 6773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:04.960351 6773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.960390 6773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:04.960417 6773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:04.967159 6773 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:45715
I20260806 06:46:04.967192 6906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:45715 every 8 connection(s)
I20260806 06:46:04.967502 6773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:04.970630 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 6773
I20260806 06:46:04.970731 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:04.970890 6373 external_mini_cluster.cc:1428] Setting key f818a2a535db4565d4a767979a956203
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:04.977141 6486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.968398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:54643 (local address 127.6.57.126:44771)
0806 06:46:04.968476 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:04.968479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:04.968489 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:04.968987 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:04.968988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:04.969002 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:04.969062 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:04.969974 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.970548 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.971960 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.972182 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.972258 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.974540 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:04.974541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:04.974541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:04.974548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:04.975796 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.976198 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.976199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.976200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.976261 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:04.976510 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:04.976511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:04.976511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:04.976629 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:04.976704 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:04.976967 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:46:04.977049 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:46:04.977329 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:04.968168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44771 (local address 127.6.57.67:54643)
0806 06:46:04.968319 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:04.968353 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:04.968868 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:04.969132 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:04.969136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:04.969683 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:04.969862 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.969867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.970658 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.970660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.971870 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:04.971873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:04.972265 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:04.972266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:04.972395 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:04.972885 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:04.972893 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:04.974418 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:04.975883 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.975889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.975892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.976107 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.976329 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:04.976330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:04.976330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:04.976434 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:04.976695 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:04.976702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:04.976835 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:46:04.976944 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:46:04.977096 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 06:46:04.977829 6907 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44771
I20260806 06:46:04.977908 6907 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:04.978101 6907 heartbeater.cc:511] Master 127.6.57.126:44771 requested a full tablet report, sending...
I20260806 06:46:04.978578 6427 ts_manager.cc:194] Registered new tserver with Master: 33eb160e7da344e99e3d382758ec59f2 (127.6.57.67:45715)
I20260806 06:46:04.979040 6427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:54643
I20260806 06:46:04.983377 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:04 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998764, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:05.009399 6915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:05.001093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.6.57.65:33645)
0806 06:46:05.001397 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:05.001402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:05.001416 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:46:05.001452 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:05.001454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:05.001470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:05.001561 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:05.002154 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:05.003066 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:05.003880 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:05.004191 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:05.004255 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:05.006618 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:05.006626 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:05.006626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:05.006633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:05.008178 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:05.008489 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:05.008492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:05.008492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:05.008570 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:05.008796 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:05.008797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:05.008798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:05.008961 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:05.009092 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:05.009184 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:46:05.009246 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":204,"threads_started":1}
W20260806 06:46:05.009822 6535 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51836
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6380](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6380](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:05.033331 6915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:05.026353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.6.57.65:33645)
0806 06:46:05.026405 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:05.026407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:05.026505 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:46:05.026618 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:05.026619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:05.026633 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:05.026685 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:05.027293 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:05.027777 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:05.028467 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:05.028694 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:05.028747 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:05.031003 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:05.031004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:05.031004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:05.031011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:05.032449 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:05.032746 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:05.032747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:05.032747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:05.032796 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:05.032952 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:05.032952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:05.032953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:05.033040 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:05.033124 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:05.033206 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:46:05.033266 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:05.033989 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 6491
I20260806 06:46:05.039479 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 6632
I20260806 06:46:05.045584 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 6773
I20260806 06:46:05.050619 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 6393
2026-08-06T06:46:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1870 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6922](info): set up 2 sockets
Aug 06 06:46:05 dist-test-slave-5l3k krb5kdc[6922](info): commencing operation
krb5kdc: starting...
W20260806 06:46:06.122879 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:06.243108 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:33617
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:33617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:06.326081 6935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:06.326282 6935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:06.326304 6935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:06.327373 6935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:06.327414 6935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:06.327428 6935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:06.327440 6935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:06.327452 6935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:06.328864 6935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:33617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:33617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:06.329113 6935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:06.329355 6935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:06.334048 6941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.334092 6940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.334048 6943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:06.334277 6935 server_base.cc:1034] running on GCE node
I20260806 06:46:06.334457 6935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:06.334728 6935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:06.335867 6935 hybrid_clock.cc:648] HybridClock initialized: now 1785998766335826 us; error 63 us; skew 500 ppm
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:06.338598 6935 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:06.339087 6935 webserver.cc:460] Webserver started at http://127.6.57.126:45721/ using document root <none> and password file <none>
I20260806 06:46:06.339334 6935 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:06.339377 6935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:06.339519 6935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:06.340518 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "242eba83a23f4851b85cfd0db66ff997"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "5a1dc7b1a133011cb4613fefe3b56c72"
server_key_iv: "6d25d3e2285af33f095e5cf55871ded1"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.340860 6935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "242eba83a23f4851b85cfd0db66ff997"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "5a1dc7b1a133011cb4613fefe3b56c72"
server_key_iv: "6d25d3e2285af33f095e5cf55871ded1"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.342196 6935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:06.343001 6950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.343189 6935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.343292 6935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "242eba83a23f4851b85cfd0db66ff997"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "5a1dc7b1a133011cb4613fefe3b56c72"
server_key_iv: "6d25d3e2285af33f095e5cf55871ded1"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.343386 6935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:06.359382 6935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:06.361691 6935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:06.361850 6935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:06.365916 6935 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:33617
I20260806 06:46:06.365948 7004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:33617 every 8 connection(s)
I20260806 06:46:06.366289 6935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:06.367214 7005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:06.368109 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 6935
I20260806 06:46:06.368202 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:06.368358 6373 external_mini_cluster.cc:1428] Setting key 7037ed9b8b192b369e4b15c5c99f4658
I20260806 06:46:06.369755 7005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997: Bootstrap starting.
I20260806 06:46:06.370515 7005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:06.370821 7005 log.cc:824] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:06.371543 7005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997: No bootstrap required, opened a new log
I20260806 06:46:06.373931 7005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } }
I20260806 06:46:06.374115 7005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:06.374156 7005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242eba83a23f4851b85cfd0db66ff997, State: Initialized, Role: FOLLOWER
I20260806 06:46:06.374298 7005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } }
I20260806 06:46:06.374377 7005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:06.374408 7005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:06.374464 7005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:06.375043 7005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } }
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:06.375134 7005 leader_election.cc:302] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242eba83a23f4851b85cfd0db66ff997; no voters:
I20260806 06:46:06.375358 7005 leader_election.cc:288] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:06.375383 7010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:06.375494 7010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [term 1 LEADER]: Becoming Leader. State: Replica: 242eba83a23f4851b85cfd0db66ff997, State: Running, Role: LEADER
I20260806 06:46:06.375578 7010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } }
I20260806 06:46:06.375629 7005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:06.375873 7010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242eba83a23f4851b85cfd0db66ff997. Latest consensus state: current_term: 1 leader_uuid: "242eba83a23f4851b85cfd0db66ff997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } } }
I20260806 06:46:06.375985 7010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:06.375888 7012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242eba83a23f4851b85cfd0db66ff997" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242eba83a23f4851b85cfd0db66ff997" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33617 } } }
I20260806 06:46:06.376248 7012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:06.376374 7016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:06.376889 7016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:06.378885 7016 catalog_manager.cc:1347] Generated new cluster ID: 986c06bfe9e44525a97a5179062b8a0a
I20260806 06:46:06.378942 7016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:06.379664 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.368880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.6.57.126:33617)
0806 06:46:06.369045 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.369053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.369078 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:46:06.369538 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.369544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.369589 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.369699 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:06.370464 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.371475 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.372529 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.372928 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.373020 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.375621 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:06.375626 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:06.375626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:06.375638 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:06.377820 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.378222 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.378225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.378225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.378351 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.378594 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.378595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.378596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.378900 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:06.379048 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.379309 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.379456 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260806 06:46:06.389163 7016 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:06.389657 7016 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:06.395557 7016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242eba83a23f4851b85cfd0db66ff997: Generated new TSK 0
I20260806 06:46:06.395730 7016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:06.427268 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:06.508128 7034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:06.508320 7034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:06.508356 7034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:06.509445 7034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:06.509485 7034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:06.509517 7034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:06.510972 7034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:06.511210 7034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:06.511432 7034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:06.516434 7043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.516408 7040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.516408 7041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:06.516610 7034 server_base.cc:1034] running on GCE node
I20260806 06:46:06.516871 7034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:06.517104 7034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:06.518296 7034 hybrid_clock.cc:648] HybridClock initialized: now 1785998766518284 us; error 28 us; skew 500 ppm
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:06.520920 7034 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:06.521378 7034 webserver.cc:460] Webserver started at http://127.6.57.65:45723/ using document root <none> and password file <none>
I20260806 06:46:06.521590 7034 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:06.521632 7034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:06.521719 7034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:06.522881 7034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "910e1246dfb44b97975fb651263b1351"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "7b8fcd3745a949bfe1a597776b6d68f2"
server_key_iv: "f11d18b34dcf22a1dcd520f1fb1b0665"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.523171 7034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "910e1246dfb44b97975fb651263b1351"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "7b8fcd3745a949bfe1a597776b6d68f2"
server_key_iv: "f11d18b34dcf22a1dcd520f1fb1b0665"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.524379 7034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:06.525066 7050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.525270 7034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:06.525372 7034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "910e1246dfb44b97975fb651263b1351"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "7b8fcd3745a949bfe1a597776b6d68f2"
server_key_iv: "f11d18b34dcf22a1dcd520f1fb1b0665"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.525458 7034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:06.537170 7034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:06.538947 7034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:06.539110 7034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:06.539311 7034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:06.539637 7034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:06.539691 7034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.539729 7034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:06.539757 7034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.545394 7034 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:45975
I20260806 06:46:06.545480 7165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:45975 every 8 connection(s)
I20260806 06:46:06.545845 7034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:06.552325 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7034
I20260806 06:46:06.552426 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:06.552624 6373 external_mini_cluster.cc:1428] Setting key 51a5e71d6f836395cb8fbd5d414742d8
I20260806 06:46:06.555107 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.546575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:40279 (local address 127.6.57.126:33617)
0806 06:46:06.546659 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.546662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.546910 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:46:06.547791 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.547792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.547803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.547852 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:06.548368 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.548842 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.549785 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.549984 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.550038 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.552048 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:06.552049 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:06.552050 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:06.552057 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:06.553689 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.554112 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.554113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.554113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.554178 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.554421 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.554422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.554422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.554580 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:06.554670 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.554934 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.555018 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:46:06.555308 7168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.546567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:33617 (local address 127.6.57.65:40279)
0806 06:46:06.546764 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.546827 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.547695 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.547902 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.547905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.548133 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:06.548301 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.548305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.548918 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.548918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.549697 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.549699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.550061 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.550062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.550156 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.550622 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:06.550630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:06.551950 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:06.553780 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.553788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.553790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.554027 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.554245 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.554246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.554247 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.554351 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.554665 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:06.554672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:06.554802 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.554910 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.555056 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":102,"threads_started":1}
I20260806 06:46:06.555744 7166 heartbeater.cc:347] Connected to a master server at 127.6.57.126:33617
I20260806 06:46:06.555831 7166 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:06.556030 7166 heartbeater.cc:511] Master 127.6.57.126:33617 requested a full tablet report, sending...
I20260806 06:46:06.556557 6968 ts_manager.cc:194] Registered new tserver with Master: 910e1246dfb44b97975fb651263b1351 (127.6.57.65:45975)
I20260806 06:46:06.557222 6968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:40279
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:06.577749 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:06.658468 7171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:06.658726 7171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:06.658751 7171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:06.659956 7171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:06.659997 7171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:06.660023 7171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:06.661561 7171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:06.661789 7171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:06.661993 7171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:06.666810 7177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.666810 7176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.666891 7179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:06.667196 7171 server_base.cc:1034] running on GCE node
I20260806 06:46:06.667409 7171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:06.667593 7171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:06.668735 7171 hybrid_clock.cc:648] HybridClock initialized: now 1785998766668718 us; error 34 us; skew 500 ppm
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:06.671187 7171 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:06.671655 7171 webserver.cc:460] Webserver started at http://127.6.57.66:41583/ using document root <none> and password file <none>
I20260806 06:46:06.671939 7171 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:06.672034 7171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:06.672170 7171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:06.673161 7171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "8de00991f477459494e084a778beea04"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "a55cda8f4f847b9d579e4863a0beaef0"
server_key_iv: "84a024540f11c1f0edc400202389563a"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.673508 7171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "8de00991f477459494e084a778beea04"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "a55cda8f4f847b9d579e4863a0beaef0"
server_key_iv: "84a024540f11c1f0edc400202389563a"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.674938 7171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:06.675657 7188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.675868 7171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:06.675976 7171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "8de00991f477459494e084a778beea04"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "a55cda8f4f847b9d579e4863a0beaef0"
server_key_iv: "84a024540f11c1f0edc400202389563a"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.676060 7171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:06.693033 7171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:06.695052 7171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:06.695212 7171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:06.695487 7171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:06.695798 7171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:06.695852 7171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.695892 7171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:06.695919 7171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.702389 7171 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:39237
I20260806 06:46:06.702463 7306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:39237 every 8 connection(s)
I20260806 06:46:06.702811 7171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:06.711760 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.703606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:58423 (local address 127.6.57.126:33617)
0806 06:46:06.703679 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.703682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.703867 (+ 185us) server_negotiation.cc:407] Connection header received
0806 06:46:06.704761 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.704762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.704772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.704823 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:06.705407 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.705872 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.706833 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.707006 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.707054 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.709187 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:06.709188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:06.709188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:06.709194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:06.710453 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.710867 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.710868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.710868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.710918 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.711169 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.711170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.711170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.711267 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:06.711318 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.711635 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.711695 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:06.711977 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.703582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:33617 (local address 127.6.57.66:58423)
0806 06:46:06.703752 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.703786 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.704671 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.704892 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.704895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.705156 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:06.705338 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.705342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.705947 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.705948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.706764 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.706766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.707077 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.707078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.707182 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.707720 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:06.707728 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:06.709108 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:06.710590 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.710596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.710598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.710803 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.710991 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.710992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.710992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.711111 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.711356 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:06.711363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:06.711490 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.711607 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.711736 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260806 06:46:06.712503 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7171
I20260806 06:46:06.712575 7307 heartbeater.cc:347] Connected to a master server at 127.6.57.126:33617
I20260806 06:46:06.712622 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:06.712689 7307 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:06.712807 6373 external_mini_cluster.cc:1428] Setting key 8f76f0a565ae51b77db462498a9484da
I20260806 06:46:06.712899 7307 heartbeater.cc:511] Master 127.6.57.126:33617 requested a full tablet report, sending...
I20260806 06:46:06.713354 6967 ts_manager.cc:194] Registered new tserver with Master: 8de00991f477459494e084a778beea04 (127.6.57.66:39237)
I20260806 06:46:06.713891 6967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:58423
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:06.739333 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:06.820765 7312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:06.820951 7312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:06.820974 7312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:06.822275 7312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:06.822324 7312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:06.822351 7312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:06.823877 7312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:33617
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:06.824131 7312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:06.824364 7312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:06.829470 7317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.829500 7320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:06.829502 7318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:06.829622 7312 server_base.cc:1034] running on GCE node
I20260806 06:46:06.829944 7312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:06.830245 7312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:06.831398 7312 hybrid_clock.cc:648] HybridClock initialized: now 1785998766831365 us; error 54 us; skew 500 ppm
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:06.834007 7312 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:06.834534 7312 webserver.cc:460] Webserver started at http://127.6.57.67:42503/ using document root <none> and password file <none>
I20260806 06:46:06.834766 7312 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:06.834810 7312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:06.834906 7312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:06.835994 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "35b0f32405d442ab96ce4a1ab9942816"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "89facd5a470d18ee9b374a0d1fb9b0fe"
server_key_iv: "4ab86b5e5612d255c52babed1134ccb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.836298 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "35b0f32405d442ab96ce4a1ab9942816"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "89facd5a470d18ee9b374a0d1fb9b0fe"
server_key_iv: "4ab86b5e5612d255c52babed1134ccb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.837572 7312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:06.838413 7328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.838696 7312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:06.838809 7312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "35b0f32405d442ab96ce4a1ab9942816"
format_stamp: "Formatted at 2026-08-06 06:46:06 on dist-test-slave-5l3k"
server_key: "89facd5a470d18ee9b374a0d1fb9b0fe"
server_key_iv: "4ab86b5e5612d255c52babed1134ccb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:06.838903 7312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:06.863437 7312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:06.866143 7312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:06.866328 7312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:06.866633 7312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:06.866994 7312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:06.867055 7312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.867102 7312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:06.867136 7312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:06.874181 7312 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:33483
I20260806 06:46:06.874260 7446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:33483 every 8 connection(s)
I20260806 06:46:06.874698 7312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:46:06 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:06.884471 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7312
I20260806 06:46:06.884574 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:06.884757 6373 external_mini_cluster.cc:1428] Setting key a3d0e7706d2732c4b11d602735939ad4
I20260806 06:46:06.885351 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.875537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36401 (local address 127.6.57.126:33617)
0806 06:46:06.875613 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.875616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.875814 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:46:06.877193 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.877194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.877217 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.877293 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:06.878052 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.878702 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.879824 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.880089 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.880167 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.882156 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:06.882157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:06.882158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:06.882168 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:06.883576 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.884096 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.884097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.884097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.884147 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.884472 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.884474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.884474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.884672 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:06.884776 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.885155 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.885236 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:46:06.885629 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.875446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:33617 (local address 127.6.57.67:36401)
0806 06:46:06.875662 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.875715 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.877054 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.877365 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.877369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.877691 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:06.877912 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.877918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.878853 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.878854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.879726 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.879730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.880175 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.880177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.880308 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.880726 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:06.880734 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:06.882036 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:06.883672 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.883678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.883680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.883923 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.884212 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:06.884213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:06.884214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:06.884348 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:06.884765 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:06.884772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:06.884943 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.885110 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.885308 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260806 06:46:06.886238 7447 heartbeater.cc:347] Connected to a master server at 127.6.57.126:33617
I20260806 06:46:06.886348 7447 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:06.886535 7447 heartbeater.cc:511] Master 127.6.57.126:33617 requested a full tablet report, sending...
I20260806 06:46:06.886902 6967 ts_manager.cc:194] Registered new tserver with Master: 35b0f32405d442ab96ce4a1ab9942816 (127.6.57.67:33483)
I20260806 06:46:06.887393 6967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:36401
I20260806 06:46:06.896661 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:46:06.903920 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.898212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.6.57.126:33617)
0806 06:46:06.898260 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.898262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.898342 (+ 80us) server_negotiation.cc:407] Connection header received
0806 06:46:06.898381 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.898382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.898391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.898446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:06.899042 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.899533 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.900215 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.900403 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.900458 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.901606 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:06.901607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:06.901608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:06.901615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:06.902942 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.903259 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.903260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.903260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.903313 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:06.903479 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:06.903479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:06.903480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:06.903629 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:06.903709 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.903791 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.903855 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:06.906253 6967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:06.908102 6967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:06.915161 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.910686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:33483 (local address 127.0.0.1:35336)
0806 06:46:06.910867 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.910909 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.910979 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.911438 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.911442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.911445 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:06.911521 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.911523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912998 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.912999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.913959 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.913961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.914883 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.914884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.914965 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.914982 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.915030 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.915078 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":68,"threads_started":1}
I20260806 06:46:06.915498 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.910792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35336 (local address 127.6.57.67:33483)
0806 06:46:06.911259 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.911263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.911272 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:06.911305 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.911307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.911321 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.911405 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:06.911608 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912873 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.914053 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.914776 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.915287 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.915364 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.915389 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.915424 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":110,"threads_started":1}
I20260806 06:46:06.915536 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.910721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:39237 (local address 127.0.0.1:60486)
0806 06:46:06.910994 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.911005 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.911060 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.911524 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.911525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.911527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:06.911603 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.911605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912722 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.912723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.914149 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.914152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.915290 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.915291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.915367 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.915370 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.915402 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.915435 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
I20260806 06:46:06.915472 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.910908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:45975 (local address 127.0.0.1:33040)
0806 06:46:06.911162 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:06.911172 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:06.911218 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:06.911619 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:06.911620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:06.911622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:06.911661 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.911662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912976 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.912977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.914159 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:06.914160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.915067 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.915069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:06.915146 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.915151 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:06.915341 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 06:46:06.915389 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":80,"threads_started":1}
I20260806 06:46:06.915773 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.911082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.6.57.66:39237)
0806 06:46:06.911297 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.911302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.911313 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:06.911355 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.911362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.911399 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.911535 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:06.911751 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912604 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.914580 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.915186 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.915517 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.915588 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.915618 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.915667 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":93,"threads_started":1}
I20260806 06:46:06.916065 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:06.911042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.6.57.65:45975)
0806 06:46:06.911208 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:06.911213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:06.911224 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:06.911299 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:06.911302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:06.911318 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:06.911405 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:06.911727 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.912856 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.914242 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:06.914970 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:06.915044 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:06.915101 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:06.915467 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 06:46:06.915508 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260806 06:46:06.916222 7377 tablet_service.cc:1467] Processing CreateTablet for tablet 41ba05a56b4d44a9b518632673aa60fb (DEFAULT_TABLE table=test-table [id=3b7798c8952f4f6984d66d2fcb99cc8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:06.916574 7377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41ba05a56b4d44a9b518632673aa60fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:06.916495 7238 tablet_service.cc:1467] Processing CreateTablet for tablet 41ba05a56b4d44a9b518632673aa60fb (DEFAULT_TABLE table=test-table [id=3b7798c8952f4f6984d66d2fcb99cc8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:06.916646 7099 tablet_service.cc:1467] Processing CreateTablet for tablet 41ba05a56b4d44a9b518632673aa60fb (DEFAULT_TABLE table=test-table [id=3b7798c8952f4f6984d66d2fcb99cc8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:06.916891 7238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41ba05a56b4d44a9b518632673aa60fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:06.916949 7099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41ba05a56b4d44a9b518632673aa60fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:06.919857 7464 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Bootstrap starting.
I20260806 06:46:06.920246 7466 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Bootstrap starting.
I20260806 06:46:06.920619 7464 tablet_bootstrap.cc:654] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:06.920792 7466 tablet_bootstrap.cc:654] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:06.921227 7464 log.cc:824] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:06.921306 7466 log.cc:824] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:06.921630 7465 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Bootstrap starting.
I20260806 06:46:06.922010 7464 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: No bootstrap required, opened a new log
I20260806 06:46:06.922106 7464 ts_tablet_manager.cc:1397] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:06.922245 7466 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: No bootstrap required, opened a new log
I20260806 06:46:06.922327 7466 ts_tablet_manager.cc:1397] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:06.922598 7465 tablet_bootstrap.cc:654] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:06.923221 7465 log.cc:824] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:06.924086 7465 tablet_bootstrap.cc:492] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: No bootstrap required, opened a new log
I20260806 06:46:06.924162 7465 ts_tablet_manager.cc:1397] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 06:46:06.924038 7464 raft_consensus.cc:348] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.924240 7464 raft_consensus.cc:374] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:06.924283 7464 raft_consensus.cc:729] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de00991f477459494e084a778beea04, State: Initialized, Role: FOLLOWER
I20260806 06:46:06.924404 7464 consensus_queue.cc:260] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.924705 7464 ts_tablet_manager.cc:1428] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:06.924849 7307 heartbeater.cc:503] Master 127.6.57.126:33617 was elected leader, sending a full tablet report...
I20260806 06:46:06.924814 7466 raft_consensus.cc:348] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.925038 7466 raft_consensus.cc:374] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:06.925091 7466 raft_consensus.cc:729] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 910e1246dfb44b97975fb651263b1351, State: Initialized, Role: FOLLOWER
I20260806 06:46:06.925246 7466 consensus_queue.cc:260] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.925575 7166 heartbeater.cc:503] Master 127.6.57.126:33617 was elected leader, sending a full tablet report...
I20260806 06:46:06.925593 7466 ts_tablet_manager.cc:1428] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:46:06.926534 7465 raft_consensus.cc:348] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.926745 7465 raft_consensus.cc:374] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:06.926802 7465 raft_consensus.cc:729] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b0f32405d442ab96ce4a1ab9942816, State: Initialized, Role: FOLLOWER
I20260806 06:46:06.926939 7465 consensus_queue.cc:260] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:06.927212 7465 ts_tablet_manager.cc:1428] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 06:46:06.927251 7447 heartbeater.cc:503] Master 127.6.57.126:33617 was elected leader, sending a full tablet report...
W20260806 06:46:06.953388 7308 tablet.cc:2369] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:07.046707 7167 tablet.cc:2369] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:07.125299 7448 tablet.cc:2369] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:07.246138 7472 raft_consensus.cc:482] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:07.246275 7472 raft_consensus.cc:504] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:07.246716 7472 leader_election.cc:288] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 910e1246dfb44b97975fb651263b1351 (127.6.57.65:45975), 8de00991f477459494e084a778beea04 (127.6.57.66:39237)
I20260806 06:46:07.249552 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.247111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:33851 (local address 127.6.57.65:45975)
0806 06:46:07.247155 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:07.247157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:07.247166 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:07.247203 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:07.247204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:07.247213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:07.247306 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:07.247482 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.247948 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.248848 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.249193 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.249279 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.249291 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:07.249443 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:46:07.249480 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:07.249552 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.246928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:45975 (local address 127.6.57.67:33851)
0806 06:46:07.246994 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:07.247007 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:07.247093 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:07.247313 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:07.247314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:07.247316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:07.247404 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:07.247405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.248036 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.248037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:07.248752 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:07.248754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.249280 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.249281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:07.249365 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.249368 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:07.249435 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 06:46:07.249476 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 06:46:07.249629 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.247023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36585 (local address 127.6.57.66:39237)
0806 06:46:07.247072 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:07.247075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:07.247249 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:46:07.247304 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:07.247306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:07.247320 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:07.247384 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:07.247535 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.247993 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.248919 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.249276 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.249361 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.249372 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:07.249514 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 06:46:07.249552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:46:07.249675 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.246946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:39237 (local address 127.6.57.67:36585)
0806 06:46:07.247162 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:07.247173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:07.247248 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:07.247385 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:07.247386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:07.247388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:07.247467 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:07.247468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.248086 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.248087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:07.248823 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:07.248825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.249363 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.249364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:07.249439 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.249443 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:07.249508 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 06:46:07.249555 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":136,"threads_started":1}
I20260806 06:46:07.250108 7260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41ba05a56b4d44a9b518632673aa60fb" candidate_uuid: "35b0f32405d442ab96ce4a1ab9942816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de00991f477459494e084a778beea04" is_pre_election: true
I20260806 06:46:07.250155 7120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41ba05a56b4d44a9b518632673aa60fb" candidate_uuid: "35b0f32405d442ab96ce4a1ab9942816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "910e1246dfb44b97975fb651263b1351" is_pre_election: true
I20260806 06:46:07.250274 7120 raft_consensus.cc:2454] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b0f32405d442ab96ce4a1ab9942816 in term 0.
I20260806 06:46:07.250274 7260 raft_consensus.cc:2454] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b0f32405d442ab96ce4a1ab9942816 in term 0.
I20260806 06:46:07.250578 7330 leader_election.cc:302] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 35b0f32405d442ab96ce4a1ab9942816, 8de00991f477459494e084a778beea04, 910e1246dfb44b97975fb651263b1351; no voters:
I20260806 06:46:07.250720 7472 raft_consensus.cc:2781] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:07.250799 7472 raft_consensus.cc:482] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:07.250818 7472 raft_consensus.cc:3037] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:07.251590 7472 raft_consensus.cc:504] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:07.251724 7472 leader_election.cc:288] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [CANDIDATE]: Term 1 election: Requested vote from peers 910e1246dfb44b97975fb651263b1351 (127.6.57.65:45975), 8de00991f477459494e084a778beea04 (127.6.57.66:39237)
I20260806 06:46:07.251878 7260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41ba05a56b4d44a9b518632673aa60fb" candidate_uuid: "35b0f32405d442ab96ce4a1ab9942816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8de00991f477459494e084a778beea04"
I20260806 06:46:07.251865 7120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41ba05a56b4d44a9b518632673aa60fb" candidate_uuid: "35b0f32405d442ab96ce4a1ab9942816" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "910e1246dfb44b97975fb651263b1351"
I20260806 06:46:07.251945 7260 raft_consensus.cc:3037] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:07.251945 7120 raft_consensus.cc:3037] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:07.252823 7260 raft_consensus.cc:2454] T 41ba05a56b4d44a9b518632673aa60fb P 8de00991f477459494e084a778beea04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b0f32405d442ab96ce4a1ab9942816 in term 1.
I20260806 06:46:07.252823 7120 raft_consensus.cc:2454] T 41ba05a56b4d44a9b518632673aa60fb P 910e1246dfb44b97975fb651263b1351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b0f32405d442ab96ce4a1ab9942816 in term 1.
I20260806 06:46:07.253034 7329 leader_election.cc:302] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b0f32405d442ab96ce4a1ab9942816, 910e1246dfb44b97975fb651263b1351; no voters:
I20260806 06:46:07.253119 7472 raft_consensus.cc:2781] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:07.253964 7472 raft_consensus.cc:686] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [term 1 LEADER]: Becoming Leader. State: Replica: 35b0f32405d442ab96ce4a1ab9942816, State: Running, Role: LEADER
I20260806 06:46:07.254073 7472 consensus_queue.cc:237] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } }
I20260806 06:46:07.255245 6968 catalog_manager.cc:5495] T 41ba05a56b4d44a9b518632673aa60fb P 35b0f32405d442ab96ce4a1ab9942816 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b0f32405d442ab96ce4a1ab9942816 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "35b0f32405d442ab96ce4a1ab9942816" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b0f32405d442ab96ce4a1ab9942816" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "910e1246dfb44b97975fb651263b1351" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 45975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8de00991f477459494e084a778beea04" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39237 } health_report { overall_health: UNKNOWN } } }
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:07.271569 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.263496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33046 (local address 127.6.57.65:45975)
0806 06:46:07.263589 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:07.263591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:07.263659 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:46:07.263735 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:07.263736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:07.263747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:07.263797 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:07.264406 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.264918 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.265707 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.265935 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.266038 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.268564 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:07.268574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:07.268575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:07.268583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:07.270265 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.270746 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.270749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.270750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.270820 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.271029 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.271030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.271030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.271179 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:07.271345 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:07.271393 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 06:46:07.271466 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998767, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998767, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:07.297125 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.289505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.6.57.65:45975)
0806 06:46:07.289583 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:07.289585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:07.289644 (+ 59us) server_negotiation.cc:407] Connection header received
0806 06:46:07.289693 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:07.289694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:07.289703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:07.289752 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:07.290268 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.290797 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.291578 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.291750 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.291795 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.294126 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:07.294127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:07.294128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:07.294135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:07.295820 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.296192 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.296194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.296194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.296253 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.296459 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.296460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.296461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.296662 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:07.296756 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:07.296974 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 06:46:07.297057 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[6922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[6922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:07.320791 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:07.313817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.6.57.65:45975)
0806 06:46:07.313878 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:07.313880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:07.313911 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:46:07.313978 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:07.313979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:07.313987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:07.314034 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:07.314680 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.315120 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.315887 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:07.316052 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:07.316100 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:07.318319 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:07.318321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:07.318321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:07.318328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:07.319742 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.320072 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.320073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.320074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.320142 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:07.320357 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:07.320357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:07.320357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:07.320439 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:07.320506 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:07.320622 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:46:07.320674 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:46:07.321391 7079 tablet_service.cc:2908] Rejecting scan request for tablet 41ba05a56b4d44a9b518632673aa60fb: Uninitialized: clean time has not yet been initialized
I20260806 06:46:07.322664 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7034
I20260806 06:46:07.328828 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7171
I20260806 06:46:07.334558 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7312
I20260806 06:46:07.339695 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 6935
2026-08-06T06:46:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2290 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[7488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[7488](info): set up 2 sockets
Aug 06 06:46:07 dist-test-slave-5l3k krb5kdc[7488](info): commencing operation
krb5kdc: starting...
W20260806 06:46:08.432583 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:08 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:08.553592 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42959
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:08.633550 7502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:08.633760 7502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:08.633782 7502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:08.635030 7502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:08.635071 7502 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:46:08.635084 7502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:08.635097 7502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:08.635109 7502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:08.635150 7502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:08.636579 7502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42959
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:08.636854 7502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:08.637071 7502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:08.641948 7510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:08.641947 7507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:08.642081 7508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:08.642297 7502 server_base.cc:1034] running on GCE node
I20260806 06:46:08.642611 7502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:08.642908 7502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:08.644047 7502 hybrid_clock.cc:648] HybridClock initialized: now 1785998768644027 us; error 38 us; skew 500 ppm
Aug 06 06:46:08 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:08.651443 7502 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:08.651899 7502 webserver.cc:460] Webserver started at http://127.6.57.126:42109/ using document root <none> and password file <none>
I20260806 06:46:08.652082 7502 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:08.652120 7502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:08.652204 7502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:08.653172 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "4f426dccdffd4317831a068a5da690f6"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "1be5f6803203bfaa846688ed99d6d445"
server_key_iv: "c1be14c722dfc57dc91a92736bf0d1f7"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.653467 7502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "4f426dccdffd4317831a068a5da690f6"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "1be5f6803203bfaa846688ed99d6d445"
server_key_iv: "c1be14c722dfc57dc91a92736bf0d1f7"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.654857 7502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:08.655582 7517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:08.655759 7502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:08.655870 7502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "4f426dccdffd4317831a068a5da690f6"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "1be5f6803203bfaa846688ed99d6d445"
server_key_iv: "c1be14c722dfc57dc91a92736bf0d1f7"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.655948 7502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:08.671991 7502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:08.673902 7502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:08.674062 7502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:08.679251 7502 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:42959
I20260806 06:46:08.679303 7582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:42959 every 8 connection(s)
I20260806 06:46:08.679616 7502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:08.680251 7583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:08.682725 7583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Bootstrap starting.
I20260806 06:46:08.683349 7583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:08.683679 7583 log.cc:824] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:08.684393 7583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: No bootstrap required, opened a new log
I20260806 06:46:08.686683 7583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } }
I20260806 06:46:08.686815 7583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:08.686859 7583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f426dccdffd4317831a068a5da690f6, State: Initialized, Role: FOLLOWER
I20260806 06:46:08.686998 7583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } }
I20260806 06:46:08.687069 7583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:08.687090 7583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:08.687157 7583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:08.687677 7583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } }
I20260806 06:46:08.687785 7583 leader_election.cc:302] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f426dccdffd4317831a068a5da690f6; no voters:
I20260806 06:46:08.687956 7583 leader_election.cc:288] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:08.688035 7586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:08.688216 7586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [term 1 LEADER]: Becoming Leader. State: Replica: 4f426dccdffd4317831a068a5da690f6, State: Running, Role: LEADER
I20260806 06:46:08.688241 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7502
I20260806 06:46:08.688226 7583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:08.688323 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:08.688318 7586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } }
I20260806 06:46:08.688489 6373 external_mini_cluster.cc:1428] Setting key 31cfdcaa18299580ae4ca2c7b3fcfe6f
I20260806 06:46:08.688625 7587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f426dccdffd4317831a068a5da690f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } } }
I20260806 06:46:08.688704 7587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:08.688958 7588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f426dccdffd4317831a068a5da690f6. Latest consensus state: current_term: 1 leader_uuid: "4f426dccdffd4317831a068a5da690f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f426dccdffd4317831a068a5da690f6" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42959 } } }
I20260806 06:46:08.689071 7588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:08.689304 7592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:08.689850 7592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:08.691520 7592 catalog_manager.cc:1347] Generated new cluster ID: 8dd3eb068fb840ffac9c169dc2b2ddc4
I20260806 06:46:08.691572 7592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:08 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:08.698756 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:08.689128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55644 (local address 127.6.57.126:42959)
0806 06:46:08.689293 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:08.689304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:08.689605 (+ 301us) server_negotiation.cc:407] Connection header received
0806 06:46:08.689750 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:08.689758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:08.689814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:08.689947 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:08.690810 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.691908 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.692687 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.693099 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.693220 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:08.695476 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:08.695480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:08.695480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:08.695508 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:08.697194 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.697582 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.697585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.697585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.697672 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.697895 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.697896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.697896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.698137 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:08.698313 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:08.698379 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:46:08.698534 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260806 06:46:08.711488 7592 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:08.712215 7592 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:08.715441 7592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Generated new TSK 0
I20260806 06:46:08.715612 7592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:08.745649 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42959
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:08.832811 7609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:08.833040 7609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:08.833069 7609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:08.834390 7609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:08.834444 7609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:08.834470 7609 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:08.834586 7609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:08.836143 7609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:08.836409 7609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:08.836668 7609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:08.842123 7614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:08.842201 7609 server_base.cc:1034] running on GCE node
W20260806 06:46:08.842274 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:08.842325 7617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:08.842674 7609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:08.843020 7609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:08.844225 7609 hybrid_clock.cc:648] HybridClock initialized: now 1785998768844204 us; error 36 us; skew 500 ppm
Aug 06 06:46:08 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:08.847044 7609 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:08.847590 7609 webserver.cc:460] Webserver started at http://127.6.57.65:33507/ using document root <none> and password file <none>
I20260806 06:46:08.847841 7609 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:08.847916 7609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:08.848104 7609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:08.849869 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "b55db78522ec3958fba7e3b7a827d12a"
server_key_iv: "140c6ca2ce5505c7d447be60f2813dae"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.850390 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "b55db78522ec3958fba7e3b7a827d12a"
server_key_iv: "140c6ca2ce5505c7d447be60f2813dae"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.852737 7609 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:08.854624 7624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:08.854785 7609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:08.854877 7609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f"
format_stamp: "Formatted at 2026-08-06 06:46:08 on dist-test-slave-5l3k"
server_key: "b55db78522ec3958fba7e3b7a827d12a"
server_key_iv: "140c6ca2ce5505c7d447be60f2813dae"
server_key_version: "encryptionkey@0"
I20260806 06:46:08.854949 7609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:08.870621 7609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:08.873210 7609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:08.873383 7609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:08.873952 7609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:08.874023 7609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:08.874080 7609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:08.874118 7609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:08.881423 7609 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:34215
Aug 06 06:46:08 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:08.881488 7746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:34215 every 8 connection(s)
I20260806 06:46:08.881819 7609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:08.884539 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:08.874588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:51105 (local address 127.6.57.126:42959)
0806 06:46:08.874651 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:08.874653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:08.874923 (+ 270us) server_negotiation.cc:407] Connection header received
0806 06:46:08.875428 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:08.875430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:08.875448 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:08.875506 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:08.876335 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.877301 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.878860 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.879321 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.879414 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:08.881972 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:08.881973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:08.881974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:08.881983 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:08.883385 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.883785 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.883786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.883786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.883835 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.884053 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.884054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.884054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.884149 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:08.884197 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:08.884415 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 06:46:08.884479 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:08.884719 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:08.874616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.65:51105)
0806 06:46:08.874798 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:08.874832 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:08.875317 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:08.875576 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:08.875579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:08.875915 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:08.876208 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:08.876215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.877416 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.877417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:08.878753 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:08.878756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.879427 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.879428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:08.879547 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:08.880080 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:08.880089 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:08.881844 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:08.883461 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:08.883467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:08.883469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:08.883717 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:08.883895 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:08.883895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:08.883896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:08.883992 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:08.884203 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:08.884208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:08.884316 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 06:46:08.884398 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:46:08.884518 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260806 06:46:08.885226 7747 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42959
I20260806 06:46:08.885309 7747 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:08.885488 7747 heartbeater.cc:511] Master 127.6.57.126:42959 requested a full tablet report, sending...
I20260806 06:46:08.886054 7533 ts_manager.cc:194] Registered new tserver with Master: c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215)
I20260806 06:46:08.887008 7533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:51105
I20260806 06:46:08.891404 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7609
I20260806 06:46:08.891525 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:08.891716 6373 external_mini_cluster.cc:1428] Setting key 9f779daf08c61372d18dc99d820dfb00
I20260806 06:46:08.892323 7753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:08.886340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.65:57175)
0806 06:46:08.886514 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:08.886526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:08.886582 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:08.886688 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:08.886689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:08.886868 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:08.886922 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:08.886924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.887714 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.887716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:08.888291 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:08.888293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.888764 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.888765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:08.888818 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:08.889272 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:08.889276 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:08.889568 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:08.891190 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:08.891191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:08.891191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:08.891340 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:08.891587 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:08.891588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:08.891588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:08.891669 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:08.891991 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:08.891992 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:08.892065 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 06:46:08.892166 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:46:08.892258 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260806 06:46:08.892360 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:08.886443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:57175 (local address 127.6.57.126:42959)
0806 06:46:08.886513 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:08.886515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:08.886590 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:46:08.886628 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:08.886629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:08.886637 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:08.886684 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:08.886985 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.887619 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.888385 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:08.888684 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:08.888753 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:08.889651 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:08.889652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:08.889652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:08.889658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:08.891122 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.891420 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.891422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.891422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.891482 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:08.891741 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:08.891742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:08.891742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:08.891861 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:08.891969 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:08.892200 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 06:46:08.892274 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:08.917842 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42959
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:08.998389 7756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:08.998665 7756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:08.998708 7756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:08.999835 7756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:08.999897 7756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:08.999930 7756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:08.999971 7756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:09.001331 7756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:09.001583 7756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:09.001788 7756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:09.006619 7763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:09.006654 7762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:09.006805 7756 server_base.cc:1034] running on GCE node
W20260806 06:46:09.006635 7765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:09.007092 7756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:09.007324 7756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:09.008476 7756 hybrid_clock.cc:648] HybridClock initialized: now 1785998769008463 us; error 35 us; skew 500 ppm
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:09.011011 7756 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:09.011523 7756 webserver.cc:460] Webserver started at http://127.6.57.66:34159/ using document root <none> and password file <none>
I20260806 06:46:09.011759 7756 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:09.011801 7756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:09.011986 7756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:09.013024 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "64e6125094b042b1a87d040ff46f30f0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "0aae6149f1f86eea4739915a32bf567a"
server_key_iv: "985befc2e35a613bf6cc89ce7cc2f7fd"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.013391 7756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "64e6125094b042b1a87d040ff46f30f0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "0aae6149f1f86eea4739915a32bf567a"
server_key_iv: "985befc2e35a613bf6cc89ce7cc2f7fd"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.014910 7756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:09.015678 7772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.015854 7756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:09.015945 7756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "64e6125094b042b1a87d040ff46f30f0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "0aae6149f1f86eea4739915a32bf567a"
server_key_iv: "985befc2e35a613bf6cc89ce7cc2f7fd"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.016013 7756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:09.035009 7756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:09.037285 7756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:09.037446 7756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:09.038028 7756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:09.038074 7756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.038095 7756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:09.038108 7756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:09.045662 7756 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:41853
I20260806 06:46:09.045753 7894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:41853 every 8 connection(s)
I20260806 06:46:09.046092 7756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:09.048028 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.038524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:42199 (local address 127.6.57.126:42959)
0806 06:46:09.038583 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.038586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.038791 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:46:09.039252 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.039254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.039269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.039354 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.039993 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.040843 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.041870 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.042201 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.042286 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.045002 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.045004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.045005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.045013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.046668 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.047069 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.047071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.047071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.047140 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.047392 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.047393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.047393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.047509 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.047585 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.047850 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.047941 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:09.048184 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.038452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.66:42199)
0806 06:46:09.038652 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.038691 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.039153 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.039420 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.039424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.039651 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:09.039910 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.039915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.040978 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.040979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.041800 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.041803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.042296 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.042297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.042432 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.043012 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:09.043020 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:09.044895 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:09.046774 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.046780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.046782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.046991 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.047207 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.047208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.047209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.047324 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.047577 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:09.047583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:09.047721 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.047825 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.047968 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":111,"threads_started":1}
I20260806 06:46:09.048646 7895 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42959
I20260806 06:46:09.048767 7895 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:09.048928 7895 heartbeater.cc:511] Master 127.6.57.126:42959 requested a full tablet report, sending...
I20260806 06:46:09.049305 7534 ts_manager.cc:194] Registered new tserver with Master: 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853)
I20260806 06:46:09.049751 7534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:42199
I20260806 06:46:09.052578 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7756
I20260806 06:46:09.052680 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:09.052843 6373 external_mini_cluster.cc:1428] Setting key 20844b63dbd244c06d13bb7018957c50
I20260806 06:46:09.055459 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.049726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:59851 (local address 127.6.57.126:42959)
0806 06:46:09.049766 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.049768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.049890 (+ 122us) server_negotiation.cc:407] Connection header received
0806 06:46:09.049935 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.049936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.049947 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.050004 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.050302 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.050808 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.051545 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.051744 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.051826 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.052804 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.052805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.052806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.052813 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.054237 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.054647 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.054648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.054648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.054697 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.054904 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.054905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.054905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.055025 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.055080 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.055259 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.055342 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:09.055474 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.049637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.66:59851)
0806 06:46:09.049772 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.049802 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.049880 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.050008 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.050008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.050170 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:09.050220 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.050221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.050903 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.050904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.051446 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.051448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.051829 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.051830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.051905 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.052332 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:09.052337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:09.052725 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:09.054321 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.054322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.054323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.054571 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.054761 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.054762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.054762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.054830 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.055089 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:09.055089 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:09.055138 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.055218 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.055334 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:09.078015 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42959
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:09.160161 7904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:09.160362 7904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:09.160385 7904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:09.161499 7904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:09.161532 7904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:09.161552 7904 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:09.161571 7904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:09.163132 7904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:09.163317 7904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:09.163486 7904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:09.168192 7911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:09.168156 7910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:09.168239 7913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:09.168401 7904 server_base.cc:1034] running on GCE node
I20260806 06:46:09.168597 7904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:09.168844 7904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:09.169987 7904 hybrid_clock.cc:648] HybridClock initialized: now 1785998769169970 us; error 32 us; skew 500 ppm
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:09.172667 7904 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:09.173089 7904 webserver.cc:460] Webserver started at http://127.6.57.67:42641/ using document root <none> and password file <none>
I20260806 06:46:09.173305 7904 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:09.173350 7904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:09.173525 7904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:09.174624 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "3b045492493943ac80b71500ec59f2b0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "1ac6aabd3f61fb36af5e3e7748f4882c"
server_key_iv: "75a21289b2b1c429e70f398a877b04cb"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.174983 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "3b045492493943ac80b71500ec59f2b0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "1ac6aabd3f61fb36af5e3e7748f4882c"
server_key_iv: "75a21289b2b1c429e70f398a877b04cb"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.176277 7904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:09.176968 7920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.177160 7904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.177253 7904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "3b045492493943ac80b71500ec59f2b0"
format_stamp: "Formatted at 2026-08-06 06:46:09 on dist-test-slave-5l3k"
server_key: "1ac6aabd3f61fb36af5e3e7748f4882c"
server_key_iv: "75a21289b2b1c429e70f398a877b04cb"
server_key_version: "encryptionkey@0"
I20260806 06:46:09.177320 7904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:09.200315 7904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:09.202097 7904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:09.202256 7904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:09.202898 7904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:09.202966 7904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.203017 7904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:09.203045 7904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.209894 7904 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:35717
I20260806 06:46:09.210005 8038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:35717 every 8 connection(s)
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:09.210304 7904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:09.212772 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 7904
I20260806 06:46:09.212844 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:09.213052 6373 external_mini_cluster.cc:1428] Setting key 30ec8097154bd11c8574145d62dea206
I20260806 06:46:09.213445 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.203402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:56109 (local address 127.6.57.126:42959)
0806 06:46:09.203473 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.203476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.203724 (+ 248us) server_negotiation.cc:407] Connection header received
0806 06:46:09.204185 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.204186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.204200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.204284 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.205102 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.205933 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.207159 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.207501 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.207605 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.210932 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.210933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.210934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.210940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.212073 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.212432 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.212433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.212433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.212485 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.212707 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.212709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.212710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.212872 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.212936 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.213265 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.213330 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:09.213624 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.203398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.67:56109)
0806 06:46:09.203568 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.203608 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.204088 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.204331 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.204334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.204702 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:09.204990 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.204997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.206059 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.206060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.207085 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.207087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.207639 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.207640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.207782 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.208935 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:09.208947 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:09.210841 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:09.212148 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.212154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.212156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.212360 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.212530 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.212530 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.212531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.212631 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.212937 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:09.212945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:09.213097 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.213211 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.213377 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260806 06:46:09.214160 8039 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42959
I20260806 06:46:09.214248 8039 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:09.214430 8039 heartbeater.cc:511] Master 127.6.57.126:42959 requested a full tablet report, sending...
I20260806 06:46:09.214823 7533 ts_manager.cc:194] Registered new tserver with Master: 3b045492493943ac80b71500ec59f2b0 (127.6.57.67:35717)
I20260806 06:46:09.215342 7533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:56109
I20260806 06:46:09.220582 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.215325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36573 (local address 127.6.57.126:42959)
0806 06:46:09.215371 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.215373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.215697 (+ 324us) server_negotiation.cc:407] Connection header received
0806 06:46:09.215766 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.215767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.215777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.215827 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.216211 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.216646 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.217306 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.217467 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.217515 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.218323 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.218324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.218324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.218330 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.219727 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.220086 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.220087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.220087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.220143 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.220263 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.220264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.220264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.220345 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.220384 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.220471 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.220525 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:46:09.220588 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.215259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.6.57.67:36573)
0806 06:46:09.215622 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.215640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.215710 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.215872 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.215873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.216076 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:09.216133 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.216135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.216728 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.216729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.217240 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.217242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.217529 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.217530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.217580 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.217969 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:09.217972 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:09.218247 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:09.219798 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.219799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.219799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.219984 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.220184 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.220185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.220185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.220230 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.220381 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:09.220382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:09.220413 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.220457 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.220528 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":68,"threads_started":1}
I20260806 06:46:09.225121 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:09.247939 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.241331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55650 (local address 127.6.57.126:42959)
0806 06:46:09.241391 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.241393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.241402 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:09.241435 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.241435 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.241444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.241493 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.242147 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.242693 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.243422 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.243595 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.243664 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.245650 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.245651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.245651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.245657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.246885 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.247208 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.247209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.247209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.247256 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.247414 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.247415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.247415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.247525 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.247599 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.247763 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.247825 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:46:09.249470 7533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:09.251194 7533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:09.257887 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.6.57.67:35717)
0806 06:46:09.254170 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.254175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.254184 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:09.254230 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.254232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.254264 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.254583 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.254641 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255330 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.256676 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257427 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257511 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257567 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.257713 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.257766 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":83,"threads_started":1}
I20260806 06:46:09.257898 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:35717 (local address 127.0.0.1:59638)
0806 06:46:09.253909 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.253951 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.254024 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.254378 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.254382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.254384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.254451 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.254463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255418 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.255419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.256578 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.256581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257519 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.257613 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257628 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.257696 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.257767 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":57,"threads_started":1}
I20260806 06:46:09.258210 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.6.57.65:34215)
0806 06:46:09.254127 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.254132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.254281 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:46:09.254323 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.254325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.254339 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.254402 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.254619 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255731 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.256949 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257449 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257536 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257595 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.258031 (+ 436us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.258113 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260806 06:46:09.258273 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:41853 (local address 127.0.0.1:53900)
0806 06:46:09.254024 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.254036 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.254122 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.254410 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.254411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.254413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.254469 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.254471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255833 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.255834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.256871 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.256874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257858 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.257928 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257932 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.258139 (+ 207us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.258186 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":85,"threads_started":1}
I20260806 06:46:09.258388 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34215 (local address 127.0.0.1:41674)
0806 06:46:09.254157 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.254166 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.254205 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.254465 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.254466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.254467 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.254539 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.254541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255822 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.255823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.256850 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.256852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257639 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.257698 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257702 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.257759 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.257805 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":9,"thread_start_us":67,"threads_started":1}
I20260806 06:46:09.258662 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.253792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.6.57.66:41853)
0806 06:46:09.253966 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.253971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.254092 (+ 121us) server_negotiation.cc:407] Connection header received
0806 06:46:09.254237 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.254240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.254259 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.254336 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.254577 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.255731 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.256965 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.257676 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.257752 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.257819 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.258405 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.258552 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":70,"threads_started":1}
I20260806 06:46:09.258792 7971 tablet_service.cc:1467] Processing CreateTablet for tablet 40564190024e4afe92d1b2b63fde6642 (DEFAULT_TABLE table=test-table [id=807cc5fafb094743b17ee3978a304748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.259047 7678 tablet_service.cc:1467] Processing CreateTablet for tablet 40564190024e4afe92d1b2b63fde6642 (DEFAULT_TABLE table=test-table [id=807cc5fafb094743b17ee3978a304748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.259205 7971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40564190024e4afe92d1b2b63fde6642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.259202 7825 tablet_service.cc:1467] Processing CreateTablet for tablet 40564190024e4afe92d1b2b63fde6642 (DEFAULT_TABLE table=test-table [id=807cc5fafb094743b17ee3978a304748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.259397 7678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40564190024e4afe92d1b2b63fde6642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.259604 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40564190024e4afe92d1b2b63fde6642. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.263381 8063 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Bootstrap starting.
I20260806 06:46:09.263949 8064 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Bootstrap starting.
I20260806 06:46:09.264287 8065 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Bootstrap starting.
I20260806 06:46:09.264433 8063 tablet_bootstrap.cc:654] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.264679 8064 tablet_bootstrap.cc:654] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.264793 8063 log.cc:824] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:09.264930 8065 tablet_bootstrap.cc:654] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.265028 8064 log.cc:824] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:09.265259 8065 log.cc:824] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:09.265559 8063 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: No bootstrap required, opened a new log
I20260806 06:46:09.265635 8063 ts_tablet_manager.cc:1397] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:09.265957 8065 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: No bootstrap required, opened a new log
I20260806 06:46:09.266021 8065 ts_tablet_manager.cc:1397] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:09.267210 8063 raft_consensus.cc:348] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.267356 8063 raft_consensus.cc:374] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.267400 8063 raft_consensus.cc:729] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b045492493943ac80b71500ec59f2b0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.267519 8063 consensus_queue.cc:260] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.267721 8065 raft_consensus.cc:348] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.267831 8063 ts_tablet_manager.cc:1428] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:09.267890 8039 heartbeater.cc:503] Master 127.6.57.126:42959 was elected leader, sending a full tablet report...
I20260806 06:46:09.267874 8065 raft_consensus.cc:374] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.267956 8065 raft_consensus.cc:729] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3c58cc4f7ac4a968a4b9a5dabd6aa4f, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.268091 8065 consensus_queue.cc:260] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.268371 8065 ts_tablet_manager.cc:1428] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:46:09.268430 7747 heartbeater.cc:503] Master 127.6.57.126:42959 was elected leader, sending a full tablet report...
I20260806 06:46:09.268805 8064 tablet_bootstrap.cc:492] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: No bootstrap required, opened a new log
I20260806 06:46:09.268877 8064 ts_tablet_manager.cc:1397] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:46:09.270354 8064 raft_consensus.cc:348] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.270527 8064 raft_consensus.cc:374] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.270579 8064 raft_consensus.cc:729] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e6125094b042b1a87d040ff46f30f0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.270694 8064 consensus_queue.cc:260] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.270959 8064 ts_tablet_manager.cc:1428] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:09.270996 7895 heartbeater.cc:503] Master 127.6.57.126:42959 was elected leader, sending a full tablet report...
W20260806 06:46:09.296728 7896 tablet.cc:2369] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:09.330713 8069 raft_consensus.cc:482] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:09.330878 8069 raft_consensus.cc:504] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.331256 8069 leader_election.cc:288] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215), 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853)
I20260806 06:46:09.334012 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.331538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:39135 (local address 127.6.57.66:41853)
0806 06:46:09.331587 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.331589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.331599 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:09.331626 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.331627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.331639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.331722 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.331888 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.332355 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333302 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.333666 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333749 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.333760 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.333915 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.333951 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:46:09.334012 7929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.331442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:41853 (local address 127.6.57.67:39135)
0806 06:46:09.331498 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.331510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.331551 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.331729 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.331731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.331732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.331813 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.331815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.332442 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.332444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.333206 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.333208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.333748 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.333830 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.333833 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.333896 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.333938 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:46:09.334199 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.331558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34613 (local address 127.6.57.65:34215)
0806 06:46:09.331602 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.331603 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.331726 (+ 123us) server_negotiation.cc:407] Connection header received
0806 06:46:09.331810 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.331811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.331821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.331880 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.332054 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.332530 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333482 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.333841 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333924 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.333932 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.334091 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.334127 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:09.334225 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.331472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34215 (local address 127.6.57.67:34613)
0806 06:46:09.331654 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.331668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.331729 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.331885 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.331886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.331888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.331982 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.331983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.332619 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.332621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.333385 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.333388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.333926 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.333927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.334012 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.334016 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.334084 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.334127 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":115,"threads_started":1}
I20260806 06:46:09.334365 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40564190024e4afe92d1b2b63fde6642" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0" is_pre_election: true
I20260806 06:46:09.334548 7846 raft_consensus.cc:2454] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 0.
I20260806 06:46:09.334604 7699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40564190024e4afe92d1b2b63fde6642" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" is_pre_election: true
I20260806 06:46:09.334790 7699 raft_consensus.cc:2454] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 0.
I20260806 06:46:09.334821 7924 leader_election.cc:302] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b045492493943ac80b71500ec59f2b0, 64e6125094b042b1a87d040ff46f30f0; no voters:
I20260806 06:46:09.334985 8069 raft_consensus.cc:2781] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:09.335047 8069 raft_consensus.cc:482] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:09.335064 8069 raft_consensus.cc:3037] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.335812 8069 raft_consensus.cc:504] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.335983 8069 leader_election.cc:288] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 election: Requested vote from peers c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215), 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853)
I20260806 06:46:09.336141 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40564190024e4afe92d1b2b63fde6642" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0"
I20260806 06:46:09.336158 7699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40564190024e4afe92d1b2b63fde6642" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f"
I20260806 06:46:09.336222 7846 raft_consensus.cc:3037] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.336222 7699 raft_consensus.cc:3037] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.337101 7846 raft_consensus.cc:2454] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 1.
I20260806 06:46:09.337101 7699 raft_consensus.cc:2454] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 1.
I20260806 06:46:09.337574 7922 leader_election.cc:302] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b045492493943ac80b71500ec59f2b0, c3c58cc4f7ac4a968a4b9a5dabd6aa4f; no voters:
I20260806 06:46:09.337755 8069 raft_consensus.cc:2781] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:09.337896 8069 raft_consensus.cc:686] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 3b045492493943ac80b71500ec59f2b0, State: Running, Role: LEADER
I20260806 06:46:09.337972 8069 consensus_queue.cc:237] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } }
I20260806 06:46:09.338811 7533 catalog_manager.cc:5495] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b045492493943ac80b71500ec59f2b0 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "3b045492493943ac80b71500ec59f2b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:09.351382 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.349252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.6.57.67:35717)
0806 06:46:09.349310 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.349329 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.349340 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:09.349421 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.349422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.349437 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.349490 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:09.349626 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.350019 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.350728 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.350888 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.350937 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.351095 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.351175 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.351229 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.351272 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.351288 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:46:09.352957 7699 raft_consensus.cc:1261] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Refusing update from remote peer 3b045492493943ac80b71500ec59f2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.352957 7846 raft_consensus.cc:1261] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Refusing update from remote peer 3b045492493943ac80b71500ec59f2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.353309 8069 consensus_queue.cc:1035] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.353462 8073 consensus_queue.cc:1035] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.355912 8076 mvcc.cc:205] Tried to move back new op lower bound from 7315450959267164160 to 7315450959209111552. Current Snapshot: MvccSnapshot[applied={T|T < 7315450959267164160}]
I20260806 06:46:09.356045 8078 mvcc.cc:205] Tried to move back new op lower bound from 7315450959267164160 to 7315450959209111552. Current Snapshot: MvccSnapshot[applied={T|T < 7315450959267164160}]
I20260806 06:46:09.356390 8077 mvcc.cc:205] Tried to move back new op lower bound from 7315450959267164160 to 7315450959209111552. Current Snapshot: MvccSnapshot[applied={T|T < 7315450959267164160}]
I20260806 06:46:09.359766 7533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:09.359874 7533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:09.360687 7533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Sending DeleteTablet for 3 replicas of tablet 40564190024e4afe92d1b2b63fde6642
I20260806 06:46:09.361003 7825 tablet_service.cc:1514] Processing DeleteTablet for tablet 40564190024e4afe92d1b2b63fde6642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260806 06:46:09.361034 7971 tablet_service.cc:1514] Processing DeleteTablet for tablet 40564190024e4afe92d1b2b63fde6642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:59638
I20260806 06:46:09.361035 7678 tablet_service.cc:1514] Processing DeleteTablet for tablet 40564190024e4afe92d1b2b63fde6642 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41674
I20260806 06:46:09.361228 7533 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:09.361384 7533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:09.361569 8086 tablet_replica.cc:330] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: stopping tablet replica
I20260806 06:46:09.361833 8087 tablet_replica.cc:330] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: stopping tablet replica
I20260806 06:46:09.361999 8087 raft_consensus.cc:2229] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:09.362108 8087 raft_consensus.cc:2258] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.362241 8086 raft_consensus.cc:2229] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:09.362542 8086 raft_consensus.cc:2258] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.362843 7825 tablet_service.cc:1467] Processing CreateTablet for tablet a4e6c33b5a2a4465837e603ae973f479 (DEFAULT_TABLE table=test-table [id=5d1e5972160044dfad99c171db855fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.362931 7678 tablet_service.cc:1467] Processing CreateTablet for tablet a4e6c33b5a2a4465837e603ae973f479 (DEFAULT_TABLE table=test-table [id=5d1e5972160044dfad99c171db855fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.362973 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e6c33b5a2a4465837e603ae973f479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.363044 7678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e6c33b5a2a4465837e603ae973f479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.363175 8086 ts_tablet_manager.cc:1905] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.364352 8065 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Bootstrap starting.
I20260806 06:46:09.364352 8064 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Bootstrap starting.
I20260806 06:46:09.364880 8086 ts_tablet_manager.cc:1918] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:09.365011 8086 log.cc:1196] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/40564190024e4afe92d1b2b63fde6642
I20260806 06:46:09.365114 8065 tablet_bootstrap.cc:654] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.364919 8064 tablet_bootstrap.cc:654] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.365470 8086 ts_tablet_manager.cc:1939] T 40564190024e4afe92d1b2b63fde6642 P 64e6125094b042b1a87d040ff46f30f0: Deleting consensus metadata
I20260806 06:46:09.365643 7971 tablet_service.cc:1467] Processing CreateTablet for tablet a4e6c33b5a2a4465837e603ae973f479 (DEFAULT_TABLE table=test-table [id=5d1e5972160044dfad99c171db855fd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:09.365796 7971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4e6c33b5a2a4465837e603ae973f479. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.366237 8087 ts_tablet_manager.cc:1905] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.366385 8065 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: No bootstrap required, opened a new log
I20260806 06:46:09.366441 8065 ts_tablet_manager.cc:1397] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:46:09.366613 8065 raft_consensus.cc:348] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.366727 8065 raft_consensus.cc:374] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.366760 8065 raft_consensus.cc:729] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3c58cc4f7ac4a968a4b9a5dabd6aa4f, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.366808 8065 consensus_queue.cc:260] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.366894 8065 ts_tablet_manager.cc:1428] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.367309 8063 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Bootstrap starting.
I20260806 06:46:09.367838 8063 tablet_bootstrap.cc:654] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.368237 8087 ts_tablet_manager.cc:1918] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:09.368358 8087 log.cc:1196] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/40564190024e4afe92d1b2b63fde6642
I20260806 06:46:09.368388 7521 catalog_manager.cc:4841] TS 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853): tablet 40564190024e4afe92d1b2b63fde6642 (table test-table [id=807cc5fafb094743b17ee3978a304748]) successfully deleted
I20260806 06:46:09.365631 8088 tablet_replica.cc:330] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: stopping tablet replica
I20260806 06:46:09.368531 8063 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: No bootstrap required, opened a new log
I20260806 06:46:09.368578 8063 ts_tablet_manager.cc:1397] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:09.368579 8088 raft_consensus.cc:2229] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:09.368732 8088 raft_consensus.cc:2258] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.368847 8087 ts_tablet_manager.cc:1939] T 40564190024e4afe92d1b2b63fde6642 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting consensus metadata
I20260806 06:46:09.368732 8063 raft_consensus.cc:348] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.368907 8063 raft_consensus.cc:374] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.368938 8063 raft_consensus.cc:729] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b045492493943ac80b71500ec59f2b0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.369021 8063 consensus_queue.cc:260] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.369117 8063 ts_tablet_manager.cc:1428] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:09.369256 8088 ts_tablet_manager.cc:1905] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.369495 8064 tablet_bootstrap.cc:492] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: No bootstrap required, opened a new log
I20260806 06:46:09.369577 8064 ts_tablet_manager.cc:1397] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 06:46:09.369731 8064 raft_consensus.cc:348] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.369792 8064 raft_consensus.cc:374] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.369810 8064 raft_consensus.cc:729] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e6125094b042b1a87d040ff46f30f0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.369853 8064 consensus_queue.cc:260] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.369938 8064 ts_tablet_manager.cc:1428] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:09.369947 7519 catalog_manager.cc:4841] TS c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215): tablet 40564190024e4afe92d1b2b63fde6642 (table test-table [id=807cc5fafb094743b17ee3978a304748]) successfully deleted
I20260806 06:46:09.370999 8088 ts_tablet_manager.cc:1918] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:09.371095 8088 log.cc:1196] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/40564190024e4afe92d1b2b63fde6642
I20260806 06:46:09.371313 8088 ts_tablet_manager.cc:1939] T 40564190024e4afe92d1b2b63fde6642 P 3b045492493943ac80b71500ec59f2b0: Deleting consensus metadata
I20260806 06:46:09.371688 7519 catalog_manager.cc:4841] TS 3b045492493943ac80b71500ec59f2b0 (127.6.57.67:35717): tablet 40564190024e4afe92d1b2b63fde6642 (table test-table [id=807cc5fafb094743b17ee3978a304748]) successfully deleted
W20260806 06:46:09.382603 7748 tablet.cc:2369] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:09.387243 8069 raft_consensus.cc:482] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:09.387328 8069 raft_consensus.cc:504] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.387465 8069 leader_election.cc:288] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215), 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853)
I20260806 06:46:09.387686 7699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e6c33b5a2a4465837e603ae973f479" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" is_pre_election: true
I20260806 06:46:09.387753 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e6c33b5a2a4465837e603ae973f479" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0" is_pre_election: true
I20260806 06:46:09.387794 7699 raft_consensus.cc:2454] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 0.
I20260806 06:46:09.387831 7846 raft_consensus.cc:2454] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 0.
I20260806 06:46:09.387938 7922 leader_election.cc:302] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b045492493943ac80b71500ec59f2b0, c3c58cc4f7ac4a968a4b9a5dabd6aa4f; no voters:
I20260806 06:46:09.388073 8069 raft_consensus.cc:2781] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:09.388110 8069 raft_consensus.cc:482] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:09.388129 8069 raft_consensus.cc:3037] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.388671 8069 raft_consensus.cc:504] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.388810 8069 leader_election.cc:288] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 election: Requested vote from peers c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215), 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853)
I20260806 06:46:09.388978 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e6c33b5a2a4465837e603ae973f479" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0"
I20260806 06:46:09.389065 7846 raft_consensus.cc:3037] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.388980 7699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4e6c33b5a2a4465837e603ae973f479" candidate_uuid: "3b045492493943ac80b71500ec59f2b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f"
I20260806 06:46:09.389123 7699 raft_consensus.cc:3037] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.389628 7699 raft_consensus.cc:2454] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 1.
I20260806 06:46:09.389770 7846 raft_consensus.cc:2454] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b045492493943ac80b71500ec59f2b0 in term 1.
I20260806 06:46:09.389827 7922 leader_election.cc:302] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b045492493943ac80b71500ec59f2b0, c3c58cc4f7ac4a968a4b9a5dabd6aa4f; no voters:
I20260806 06:46:09.389920 8069 raft_consensus.cc:2781] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:09.389964 8069 raft_consensus.cc:686] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 1 LEADER]: Becoming Leader. State: Replica: 3b045492493943ac80b71500ec59f2b0, State: Running, Role: LEADER
I20260806 06:46:09.390005 8069 consensus_queue.cc:237] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.390471 7533 catalog_manager.cc:5495] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b045492493943ac80b71500ec59f2b0 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "3b045492493943ac80b71500ec59f2b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } health_report { overall_health: HEALTHY } } }
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998768, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:09.419457 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.411294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42959 (local address 127.0.0.1:55656)
0806 06:46:09.411501 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.411512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.411572 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.411725 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.411726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.412051 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:09.412101 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.412103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.413095 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.413097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.413651 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.413653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.414170 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.414171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.414226 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.414865 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:09.414869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:09.416475 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:09.418200 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.418208 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.418211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.418426 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.418759 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:09.418761 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:09.418761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:09.418835 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:09.419113 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:09.419122 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:09.419222 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.419288 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.419383 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":101,"threads_started":1}
I20260806 06:46:09.419559 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.411355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55656 (local address 127.6.57.126:42959)
0806 06:46:09.411436 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.411439 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.411583 (+ 144us) server_negotiation.cc:407] Connection header received
0806 06:46:09.411640 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.411642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.411653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.411712 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.412177 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.412953 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.413745 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.414075 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.414170 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.416568 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.416569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.416570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.416578 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.418107 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.418554 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.418556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.418556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.418620 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.418908 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.418909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.418909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.419037 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.419104 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.419317 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.419447 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":47}
I20260806 06:46:09.421694 7533 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:55656:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:46:09.421936 7533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:09.423444 7678 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3dba2cbbda484fa403c87812adcf69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e6128edbea4274bc9486338cd008ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:09.423475 7825 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3dba2cbbda484fa403c87812adcf69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e6128edbea4274bc9486338cd008ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:09.423621 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3dba2cbbda484fa403c87812adcf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.423623 7678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3dba2cbbda484fa403c87812adcf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.424957 8065 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Bootstrap starting.
I20260806 06:46:09.425056 7971 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3dba2cbbda484fa403c87812adcf69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d0e6128edbea4274bc9486338cd008ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:09.425134 8064 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: Bootstrap starting.
I20260806 06:46:09.425194 7971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3dba2cbbda484fa403c87812adcf69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:09.425361 8065 tablet_bootstrap.cc:654] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.425572 8064 tablet_bootstrap.cc:654] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.426260 8064 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: No bootstrap required, opened a new log
I20260806 06:46:09.426317 8064 ts_tablet_manager.cc:1397] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:09.426504 8064 raft_consensus.cc:348] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.426575 8064 raft_consensus.cc:374] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.426594 8064 raft_consensus.cc:729] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e6125094b042b1a87d040ff46f30f0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.426633 8064 consensus_queue.cc:260] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.426719 8064 ts_tablet_manager.cc:1428] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:09.426676 8063 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: Bootstrap starting.
I20260806 06:46:09.426851 8071 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.426932 8071 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.427021 8063 tablet_bootstrap.cc:654] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:09.427594 8063 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: No bootstrap required, opened a new log
I20260806 06:46:09.427645 8063 ts_tablet_manager.cc:1397] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:09.427750 8063 raft_consensus.cc:348] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.427780 8065 tablet_bootstrap.cc:492] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: No bootstrap required, opened a new log
I20260806 06:46:09.427816 8063 raft_consensus.cc:374] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.427829 8065 ts_tablet_manager.cc:1397] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 06:46:09.427843 8063 raft_consensus.cc:729] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b045492493943ac80b71500ec59f2b0, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.427887 8063 consensus_queue.cc:260] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.427970 8063 ts_tablet_manager.cc:1428] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.427992 8065 raft_consensus.cc:348] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.427991 8069 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.428059 8065 raft_consensus.cc:374] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:09.428067 8069 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.428081 8065 raft_consensus.cc:729] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3c58cc4f7ac4a968a4b9a5dabd6aa4f, State: Initialized, Role: FOLLOWER
I20260806 06:46:09.428140 8065 consensus_queue.cc:260] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.428231 8065 ts_tablet_manager.cc:1428] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:09.428274 8070 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.428383 8070 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 06:46:09.460891 8040 tablet.cc:2369] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:09.623840 8070 raft_consensus.cc:482] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:09.623960 8070 raft_consensus.cc:504] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.624347 8070 leader_election.cc:288] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853), 3b045492493943ac80b71500ec59f2b0 (127.6.57.67:35717)
I20260806 06:46:09.628301 8060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.624558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:43067 (local address 127.6.57.66:41853)
0806 06:46:09.624606 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.624608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.624753 (+ 145us) server_negotiation.cc:407] Connection header received
0806 06:46:09.624789 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.624790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.624801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.624857 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.625076 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.625815 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.627364 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.627902 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.627978 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.627989 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.628192 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.628229 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:09.628301 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.624644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:37805 (local address 127.6.57.67:35717)
0806 06:46:09.624752 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.624755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.624782 (+ 27us) server_negotiation.cc:407] Connection header received
0806 06:46:09.624812 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.624813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.624821 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.624870 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:09.625082 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.625815 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.627366 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.627902 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.627978 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.627989 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.628192 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.628229 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 06:46:09.628368 8104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.624552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:35717 (local address 127.6.57.65:37805)
0806 06:46:09.624667 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.624678 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.624735 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.624889 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.624890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.624892 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.624975 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.624977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.625915 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.625916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.627269 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.627273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.628027 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.628028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.628091 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.628098 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.628168 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.628214 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":107,"thread_start_us":59,"threads_started":1}
I20260806 06:46:09.628372 8103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.624504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:41853 (local address 127.6.57.65:43067)
0806 06:46:09.624666 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.624678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.624736 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.624889 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.624890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.624894 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:09.624975 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.624978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.625916 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.625916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.627269 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.627273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.628017 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.628018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.628091 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.628099 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.628165 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.628214 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":53,"thread_start_us":91,"threads_started":1}
I20260806 06:46:09.628703 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3dba2cbbda484fa403c87812adcf69" candidate_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0" is_pre_election: true
I20260806 06:46:09.628727 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3dba2cbbda484fa403c87812adcf69" candidate_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b045492493943ac80b71500ec59f2b0" is_pre_election: true
I20260806 06:46:09.628809 7846 raft_consensus.cc:2454] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3c58cc4f7ac4a968a4b9a5dabd6aa4f in term 0.
I20260806 06:46:09.628860 7989 raft_consensus.cc:2454] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3c58cc4f7ac4a968a4b9a5dabd6aa4f in term 0.
I20260806 06:46:09.629000 7628 leader_election.cc:302] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e6125094b042b1a87d040ff46f30f0, c3c58cc4f7ac4a968a4b9a5dabd6aa4f; no voters:
I20260806 06:46:09.629170 8070 raft_consensus.cc:2781] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:09.629225 8070 raft_consensus.cc:482] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:09.629248 8070 raft_consensus.cc:3037] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.630009 8070 raft_consensus.cc:504] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.630151 8070 leader_election.cc:288] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [CANDIDATE]: Term 1 election: Requested vote from peers 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853), 3b045492493943ac80b71500ec59f2b0 (127.6.57.67:35717)
I20260806 06:46:09.630316 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3dba2cbbda484fa403c87812adcf69" candidate_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b045492493943ac80b71500ec59f2b0"
I20260806 06:46:09.630344 7846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3dba2cbbda484fa403c87812adcf69" candidate_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6125094b042b1a87d040ff46f30f0"
I20260806 06:46:09.630412 7846 raft_consensus.cc:3037] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.630415 7989 raft_consensus.cc:3037] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:09.630995 7846 raft_consensus.cc:2454] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3c58cc4f7ac4a968a4b9a5dabd6aa4f in term 1.
I20260806 06:46:09.631181 7628 leader_election.cc:302] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64e6125094b042b1a87d040ff46f30f0, c3c58cc4f7ac4a968a4b9a5dabd6aa4f; no voters:
I20260806 06:46:09.631299 7989 raft_consensus.cc:2454] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3c58cc4f7ac4a968a4b9a5dabd6aa4f in term 1.
I20260806 06:46:09.631297 8070 raft_consensus.cc:2781] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:09.631408 8070 raft_consensus.cc:686] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 LEADER]: Becoming Leader. State: Replica: c3c58cc4f7ac4a968a4b9a5dabd6aa4f, State: Running, Role: LEADER
I20260806 06:46:09.631516 8070 consensus_queue.cc:237] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } }
I20260806 06:46:09.631882 8105 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: TxnStatusTablet state changed. Reason: New leader c3c58cc4f7ac4a968a4b9a5dabd6aa4f. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.631975 8105 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:09.632218 8107 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:46:09.632342 8107 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:46:09.632254 7533 catalog_manager.cc:5495] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f reported cstate change: term changed from 0 to 1, leader changed from <none> to c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65). New cstate: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:09.654680 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.651890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.6.57.65:34215)
0806 06:46:09.651946 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.651948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.651979 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:46:09.652018 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.652018 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.652036 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.652094 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:09.652251 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.652720 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.653829 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.654008 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.654074 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.654290 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.654388 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.654463 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.654591 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.654624 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:46:09.654721 8099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.651775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34215 (local address 127.0.0.1:41688)
0806 06:46:09.651866 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.651880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.651932 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.652127 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.652128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.652129 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:09.652182 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.652185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.652800 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.652801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.653738 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.653741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.654095 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.654096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.654163 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.654180 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.654495 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.654513 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.654557 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.654623 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:46:09.656023 7846 raft_consensus.cc:1261] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Refusing update from remote peer c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.656176 7989 raft_consensus.cc:1261] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Refusing update from remote peer c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.656286 8070 consensus_queue.cc:1035] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.656410 8105 consensus_queue.cc:1035] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.657021 8069 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: TxnStatusTablet state changed. Reason: New leader c3c58cc4f7ac4a968a4b9a5dabd6aa4f. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.657120 8069 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.657351 8071 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: TxnStatusTablet state changed. Reason: New leader c3c58cc4f7ac4a968a4b9a5dabd6aa4f. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.657441 8071 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.658017 8105 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.658119 8105 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:09.658324 8105 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.658401 8105 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:09.658710 8071 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.658795 8071 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 64e6125094b042b1a87d040ff46f30f0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.658943 8069 tablet_replica.cc:439] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 } } peers { permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 } } peers { permanent_uuid: "3b045492493943ac80b71500ec59f2b0" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35717 } } }
I20260806 06:46:09.659022 8069 tablet_replica.cc:442] T 5a3dba2cbbda484fa403c87812adcf69 P 3b045492493943ac80b71500ec59f2b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:09.664530 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.662286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:56465 (local address 127.6.57.65:34215)
0806 06:46:09.662333 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.662335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.662346 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:09.662413 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.662414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.662425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.662513 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:09.662647 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.663083 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.663828 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.664033 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.664114 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.664286 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.664325 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.664387 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.664452 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.664469 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:46:09.664566 8045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.662219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34215 (local address 127.6.57.67:56465)
0806 06:46:09.662302 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.662315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.662360 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.662516 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.662517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.662518 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:09.662562 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.662564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.663169 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.663170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.663739 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.663741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.664120 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.664121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.664178 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.664193 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.664388 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.664403 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.664449 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.664496 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:46:09.666684 7846 raft_consensus.cc:1261] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Refusing update from remote peer 3b045492493943ac80b71500ec59f2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.666715 7699 raft_consensus.cc:1261] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Refusing update from remote peer 3b045492493943ac80b71500ec59f2b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:09.666903 8069 consensus_queue.cc:1035] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e6125094b042b1a87d040ff46f30f0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 41853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.667014 8073 consensus_queue.cc:1035] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3c58cc4f7ac4a968a4b9a5dabd6aa4f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:09.679196 8059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.675263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:47935 (local address 127.6.57.67:35717)
0806 06:46:09.675313 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.675316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.676174 (+ 858us) server_negotiation.cc:407] Connection header received
0806 06:46:09.676254 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.676255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.676267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.676321 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:09.676462 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.677186 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.678347 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.678597 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.678665 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.678866 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.678903 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.678956 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.679088 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.679109 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:09.679219 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.675231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:35717 (local address 127.6.57.65:47935)
0806 06:46:09.676104 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:09.676124 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:09.676199 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:09.676329 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:09.676330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:09.676331 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:09.676387 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.676389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.677284 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.677285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.678271 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:09.678274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.678664 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.678665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:09.678737 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.678754 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:09.679019 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:09.679021 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:09.679081 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:46:09.679135 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"thread_start_us":119,"threads_started":1}
I20260806 06:46:09.686805 7532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:09.686884 7532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:09.687471 7532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f426dccdffd4317831a068a5da690f6: Sending DeleteTablet for 3 replicas of tablet a4e6c33b5a2a4465837e603ae973f479
I20260806 06:46:09.687718 7678 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e6c33b5a2a4465837e603ae973f479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41674
I20260806 06:46:09.687731 7971 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e6c33b5a2a4465837e603ae973f479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:59638
I20260806 06:46:09.687745 7825 tablet_service.cc:1514] Processing DeleteTablet for tablet a4e6c33b5a2a4465837e603ae973f479 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:09 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53900
I20260806 06:46:09.687804 8087 tablet_replica.cc:330] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: stopping tablet replica
I20260806 06:46:09.687819 8086 tablet_replica.cc:330] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: stopping tablet replica
I20260806 06:46:09.687816 8088 tablet_replica.cc:330] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: stopping tablet replica
I20260806 06:46:09.687848 8087 raft_consensus.cc:2229] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:09.687858 8086 raft_consensus.cc:2229] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:09.687876 8088 raft_consensus.cc:2229] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:09.687892 8087 raft_consensus.cc:2258] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.687906 8086 raft_consensus.cc:2258] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.687922 8088 raft_consensus.cc:2258] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:09.688310 8087 ts_tablet_manager.cc:1905] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.689775 8088 ts_tablet_manager.cc:1905] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.689924 8087 ts_tablet_manager.cc:1918] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:09.689985 8087 log.cc:1196] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/a4e6c33b5a2a4465837e603ae973f479
I20260806 06:46:09.690215 8087 ts_tablet_manager.cc:1939] T a4e6c33b5a2a4465837e603ae973f479 P c3c58cc4f7ac4a968a4b9a5dabd6aa4f: Deleting consensus metadata
I20260806 06:46:09.690536 7519 catalog_manager.cc:4841] TS c3c58cc4f7ac4a968a4b9a5dabd6aa4f (127.6.57.65:34215): tablet a4e6c33b5a2a4465837e603ae973f479 (table test-table [id=5d1e5972160044dfad99c171db855fd3]) successfully deleted
I20260806 06:46:09.691241 8088 ts_tablet_manager.cc:1918] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:09.691303 8088 log.cc:1196] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/a4e6c33b5a2a4465837e603ae973f479
I20260806 06:46:09.691370 8086 ts_tablet_manager.cc:1905] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:09.691545 8088 ts_tablet_manager.cc:1939] T a4e6c33b5a2a4465837e603ae973f479 P 3b045492493943ac80b71500ec59f2b0: Deleting consensus metadata
I20260806 06:46:09.691828 7519 catalog_manager.cc:4841] TS 3b045492493943ac80b71500ec59f2b0 (127.6.57.67:35717): tablet a4e6c33b5a2a4465837e603ae973f479 (table test-table [id=5d1e5972160044dfad99c171db855fd3]) successfully deleted
I20260806 06:46:09.692849 8086 ts_tablet_manager.cc:1918] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:09.692899 8086 log.cc:1196] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/a4e6c33b5a2a4465837e603ae973f479
I20260806 06:46:09.693100 8086 ts_tablet_manager.cc:1939] T a4e6c33b5a2a4465837e603ae973f479 P 64e6125094b042b1a87d040ff46f30f0: Deleting consensus metadata
I20260806 06:46:09.693464 7521 catalog_manager.cc:4841] TS 64e6125094b042b1a87d040ff46f30f0 (127.6.57.66:41853): tablet a4e6c33b5a2a4465837e603ae973f479 (table test-table [id=5d1e5972160044dfad99c171db855fd3]) successfully deleted
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998769, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:09.701798 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.693222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.6.57.65:34215)
0806 06:46:09.693381 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.693384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.693396 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:09.693421 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.693421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.693432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.693497 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.694228 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.695022 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.695715 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.696013 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.696084 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.699179 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.699187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.699187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.699194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.700718 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.701106 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.701109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.701109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.701159 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.701354 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.701355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.701355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.701472 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.701563 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.701682 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.701740 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
W20260806 06:46:09.702077 7740 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41692
I20260806 06:46:09.709486 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:09.703119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.6.57.126:42959)
0806 06:46:09.703165 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:09.703167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:09.703246 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:46:09.703311 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:09.703312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:09.703320 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:09.703364 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:09.703951 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.704414 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.705325 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:09.705493 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:09.705540 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:09.706929 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:09.706930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:09.706931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:09.706937 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:09.708424 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.708837 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.708838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.708838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.708884 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:09.709080 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:09.709080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:09.709081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:09.709170 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:09.709239 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:09.709376 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 06:46:09.709428 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 06:46:09.709857 7532 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55670
I20260806 06:46:09.710495 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7609
I20260806 06:46:09.718006 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7756
I20260806 06:46:09.724575 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7904
I20260806 06:46:09.730540 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 7502
2026-08-06T06:46:09Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2390 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[8127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[8127](info): set up 2 sockets
Aug 06 06:46:09 dist-test-slave-5l3k krb5kdc[8127](info): commencing operation
krb5kdc: starting...
W20260806 06:46:10.788518 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:10 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:10.911653 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37923
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:10.994053 8140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:10.994251 8140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:10.994272 8140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:10.995437 8140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:10.995479 8140 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:46:10.995492 8140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:10.995538 8140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:10.995570 8140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:10.995586 8140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:10.997069 8140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:37923
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:10.997323 8140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:10.997550 8140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:11.002329 8147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.002317 8146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:11.002370 8140 server_base.cc:1034] running on GCE node
W20260806 06:46:11.002305 8149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:11.002698 8140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:11.002931 8140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:11.004081 8140 hybrid_clock.cc:648] HybridClock initialized: now 1785998771004052 us; error 44 us; skew 500 ppm
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:11.006584 8140 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:11.006956 8140 webserver.cc:460] Webserver started at http://127.6.57.126:46175/ using document root <none> and password file <none>
I20260806 06:46:11.007148 8140 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:11.007189 8140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:11.007273 8140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:11.008224 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "caa62337afa541b2bf494f1b42da5d57"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "aeb64703027d3d6fd7ad4a61c6637cf0"
server_key_iv: "c722565c0549fc4be35ae888904d73a9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.008519 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "caa62337afa541b2bf494f1b42da5d57"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "aeb64703027d3d6fd7ad4a61c6637cf0"
server_key_iv: "c722565c0549fc4be35ae888904d73a9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.009836 8140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:11.010639 8156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.010811 8140 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:11.010908 8140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "caa62337afa541b2bf494f1b42da5d57"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "aeb64703027d3d6fd7ad4a61c6637cf0"
server_key_iv: "c722565c0549fc4be35ae888904d73a9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.010967 8140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:11.027768 8140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:11.029688 8140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:11.029808 8140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:11.034893 8140 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:37923
I20260806 06:46:11.034986 8221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:37923 every 8 connection(s)
I20260806 06:46:11.035331 8140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:11.036073 8222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.036595 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8140
I20260806 06:46:11.036700 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:11.036863 6373 external_mini_cluster.cc:1428] Setting key 849c6d2928571745fd87604bec4956da
I20260806 06:46:11.039369 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Bootstrap starting.
I20260806 06:46:11.039974 8222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.040277 8222 log.cc:824] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:11.040951 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: No bootstrap required, opened a new log
I20260806 06:46:11.042830 8222 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } }
I20260806 06:46:11.042956 8222 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.042979 8222 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa62337afa541b2bf494f1b42da5d57, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.043100 8222 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } }
I20260806 06:46:11.043186 8222 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:11.043210 8222 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:11.043273 8222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.043915 8222 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } }
I20260806 06:46:11.044014 8222 leader_election.cc:302] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa62337afa541b2bf494f1b42da5d57; no voters:
I20260806 06:46:11.044199 8222 leader_election.cc:288] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:11.044265 8227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:11.044418 8227 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [term 1 LEADER]: Becoming Leader. State: Replica: caa62337afa541b2bf494f1b42da5d57, State: Running, Role: LEADER
I20260806 06:46:11.044464 8222 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:11.044507 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } }
I20260806 06:46:11.044873 8229 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa62337afa541b2bf494f1b42da5d57. Latest consensus state: current_term: 1 leader_uuid: "caa62337afa541b2bf494f1b42da5d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } } }
I20260806 06:46:11.044965 8229 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:11.045173 8228 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa62337afa541b2bf494f1b42da5d57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa62337afa541b2bf494f1b42da5d57" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37923 } } }
I20260806 06:46:11.045310 8228 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57 [sys.catalog]: This master's current role is: LEADER
W20260806 06:46:11.046339 8243 catalog_manager.cc:1558] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:11.046437 8243 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:11.046999 8233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:11.047200 8233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:11.048461 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.037474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.6.57.126:37923)
0806 06:46:11.037670 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.037680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.037720 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:46:11.038399 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.038414 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.038467 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.038668 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.039478 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.040493 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.041297 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.041697 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.041810 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.044478 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.044481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.044482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.044494 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.046709 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.047104 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.047107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.047107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.047210 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.047422 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.047424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.047424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.047737 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.047881 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.048111 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.048244 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260806 06:46:11.049018 8233 catalog_manager.cc:1347] Generated new cluster ID: ca73f61c6e314581acc0c0c7264e58cc
I20260806 06:46:11.049078 8233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:11.058046 8233 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:11.058744 8233 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:11.065968 8233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Generated new TSK 0
I20260806 06:46:11.066129 8233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:11.093351 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37923
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:11.174942 8248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:11.175172 8248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:11.175210 8248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:11.176355 8248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:11.176424 8248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:11.176460 8248 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:46:11.176489 8248 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:11.176533 8248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:11.178007 8248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:37923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:11.178293 8248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:11.178556 8248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:11.183548 8256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.183622 8254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.183549 8253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:11.183761 8248 server_base.cc:1034] running on GCE node
I20260806 06:46:11.183951 8248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:11.184149 8248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:11.185287 8248 hybrid_clock.cc:648] HybridClock initialized: now 1785998771185280 us; error 30 us; skew 500 ppm
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:11.187937 8248 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:11.188413 8248 webserver.cc:460] Webserver started at http://127.6.57.65:42389/ using document root <none> and password file <none>
I20260806 06:46:11.188659 8248 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:11.188705 8248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:11.188879 8248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:11.189903 8248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "8285860a6d3941d9b2c755ec470f0d9d"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "51b9e1e2eb8fe7045f9ad5f127e1130f"
server_key_iv: "9942838ebb56eff6aa3590913992bff9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.190275 8248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "8285860a6d3941d9b2c755ec470f0d9d"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "51b9e1e2eb8fe7045f9ad5f127e1130f"
server_key_iv: "9942838ebb56eff6aa3590913992bff9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.191744 8248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:46:11.192552 8263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.192781 8248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:11.192878 8248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "8285860a6d3941d9b2c755ec470f0d9d"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "51b9e1e2eb8fe7045f9ad5f127e1130f"
server_key_iv: "9942838ebb56eff6aa3590913992bff9"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.192952 8248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:11.203905 8248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:11.206166 8248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:11.206341 8248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:11.206931 8248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:11.206991 8248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.207031 8248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:11.207050 8248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.214533 8248 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:38531
I20260806 06:46:11.214931 8248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:11.215358 8382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:38531 every 8 connection(s)
I20260806 06:46:11.217437 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.207268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:49803 (local address 127.6.57.126:37923)
0806 06:46:11.207311 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.207314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.207597 (+ 283us) server_negotiation.cc:407] Connection header received
0806 06:46:11.208342 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.208343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.208354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.208397 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.209300 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.210205 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.211364 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.211593 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.211675 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.214719 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.214720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.214720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.214727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.216253 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.216608 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.216609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.216610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.216661 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.216873 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.216873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.216874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.216970 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.217020 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.217295 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.217373 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:11.217496 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.207259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.65:49803)
0806 06:46:11.207440 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.207483 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.208233 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.208470 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.208473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.208898 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.209190 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.209196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.210314 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.210315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.211262 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.211266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.211696 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.211697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.211830 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.212339 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.212354 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.214637 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.216324 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.216328 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.216330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.216534 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.216719 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.216719 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.216720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.216819 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.217024 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.217029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.217150 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.217232 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.217347 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260806 06:46:11.218045 8383 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37923
I20260806 06:46:11.218176 8383 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:11.218390 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8248
I20260806 06:46:11.218578 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:11.218559 8383 heartbeater.cc:511] Master 127.6.57.126:37923 requested a full tablet report, sending...
I20260806 06:46:11.218953 6373 external_mini_cluster.cc:1428] Setting key 7b93cbc8c1a5cd2e75b0ffdb0dcb3925
I20260806 06:46:11.219578 8172 ts_manager.cc:194] Registered new tserver with Master: 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531)
I20260806 06:46:11.221016 8172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:49803
I20260806 06:46:11.230315 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.220008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:47335 (local address 127.6.57.126:37923)
0806 06:46:11.220069 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.220071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.220308 (+ 237us) server_negotiation.cc:407] Connection header received
0806 06:46:11.220441 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.220441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.220464 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.220556 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.221043 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.221905 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.223485 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.223982 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.224147 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.225505 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.225507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.225507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.225532 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.228163 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.228879 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.228880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.228880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.228983 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.229337 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.229338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.229338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.229501 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.229741 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.230027 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.230159 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:11.230441 8389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.219919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.65:47335)
0806 06:46:11.220196 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.220221 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.220334 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.220563 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.220575 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.220829 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.220902 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.220909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.222051 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.222059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.223375 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.223378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.224132 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.224132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.224215 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.224851 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.224864 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.225378 (+ 514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.228335 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.228337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.228337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.228727 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.229112 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.229113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.229114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.229208 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.229733 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.229735 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.229835 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.229978 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.230162 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":156,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:11.258447 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37923
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:11.383688 8392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:11.383876 8392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:11.383900 8392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:11.384972 8392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:11.385004 8392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:11.385025 8392 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:46:11.385036 8392 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:11.385052 8392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:11.386377 8392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:37923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:11.386651 8392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:11.386838 8392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:11.391882 8397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.391883 8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.391937 8398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:11.392241 8392 server_base.cc:1034] running on GCE node
I20260806 06:46:11.392452 8392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:11.392700 8392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:11.393864 8392 hybrid_clock.cc:648] HybridClock initialized: now 1785998771393865 us; error 52 us; skew 500 ppm
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:11.396327 8392 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:11.396814 8392 webserver.cc:460] Webserver started at http://127.6.57.66:34117/ using document root <none> and password file <none>
I20260806 06:46:11.397042 8392 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:11.397104 8392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:11.397236 8392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:11.398259 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "197fd40dcd134633a18ae59d054a9d3e"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "a93f37588faec5f10fb983b201904bab"
server_key_iv: "118babe6a9dcd5ea471457000e3e8f6a"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.398645 8392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "197fd40dcd134633a18ae59d054a9d3e"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "a93f37588faec5f10fb983b201904bab"
server_key_iv: "118babe6a9dcd5ea471457000e3e8f6a"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.400027 8392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:11.400828 8407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.401008 8392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:11.401105 8392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "197fd40dcd134633a18ae59d054a9d3e"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "a93f37588faec5f10fb983b201904bab"
server_key_iv: "118babe6a9dcd5ea471457000e3e8f6a"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.401188 8392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:11.420169 8392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:11.422237 8392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:11.422407 8392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:11.423116 8392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:11.423171 8392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.423207 8392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:11.423257 8392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.430541 8392 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:36803
I20260806 06:46:11.430671 8527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:36803 every 8 connection(s)
I20260806 06:46:11.430994 8392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:11.432834 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.423515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:33493 (local address 127.6.57.126:37923)
0806 06:46:11.423587 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.423590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.423834 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:46:11.424397 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.424398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.424411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.424460 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.425364 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.426236 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.427361 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.427723 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.427807 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.429973 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.429975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.429975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.429984 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.431653 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.432030 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.432031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.432032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.432100 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.432309 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.432309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.432310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.432405 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.432453 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.432711 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.432771 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:46:11.433063 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.423520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.66:33493)
0806 06:46:11.423694 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.423735 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.424282 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.424537 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.424541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.424938 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.425243 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.425252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.426347 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.426349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.427249 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.427252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.427838 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.427840 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.427965 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.428473 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.428483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.429871 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.431725 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.431729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.431731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.431962 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.432160 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.432160 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.432160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.432256 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.432458 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.432462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.432582 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.432666 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.432811 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":104,"threads_started":1}
I20260806 06:46:11.433563 8528 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37923
I20260806 06:46:11.433696 8528 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:11.433892 8528 heartbeater.cc:511] Master 127.6.57.126:37923 requested a full tablet report, sending...
I20260806 06:46:11.434449 8173 ts_manager.cc:194] Registered new tserver with Master: 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803)
I20260806 06:46:11.435032 8173 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:33493
I20260806 06:46:11.436235 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8392
I20260806 06:46:11.436337 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:11.436548 6373 external_mini_cluster.cc:1428] Setting key 83151d72a584efdb2593a9982bba6181
I20260806 06:46:11.441134 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.434764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:46817 (local address 127.6.57.126:37923)
0806 06:46:11.434805 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.434807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.434970 (+ 163us) server_negotiation.cc:407] Connection header received
0806 06:46:11.435023 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.435024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.435034 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.435115 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.435541 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.436125 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.437148 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.437360 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.437413 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.438580 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.438581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.438581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.438587 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.439937 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.440371 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.440372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.440373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.440422 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.440636 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.440637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.440637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.440778 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.440830 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.441017 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.441076 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:11.441282 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.434706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.66:46817)
0806 06:46:11.434876 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.434886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.434960 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.435141 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.435142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.435365 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.435448 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.435450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.436236 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.436238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.437065 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.437068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.437442 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.437443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.437543 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.437993 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.437999 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.438455 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.440038 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.440039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.440040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.440293 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.440488 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.440489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.440490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.440570 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.440851 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.440852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.440903 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.441002 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.441124 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:11.463560 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37923
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:11.547449 8537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:11.547650 8537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:11.547675 8537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:11.548784 8537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:11.548823 8537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:11.548843 8537 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 06:46:11.548856 8537 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:11.548871 8537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:11.550246 8537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:37923
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:11.550426 8537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:11.550662 8537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:11.555661 8542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.555613 8543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:11.555619 8545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:11.555887 8537 server_base.cc:1034] running on GCE node
I20260806 06:46:11.556063 8537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:11.556257 8537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:11.557432 8537 hybrid_clock.cc:648] HybridClock initialized: now 1785998771557400 us; error 47 us; skew 500 ppm
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:11.559926 8537 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:11.560392 8537 webserver.cc:460] Webserver started at http://127.6.57.67:38683/ using document root <none> and password file <none>
I20260806 06:46:11.560624 8537 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:11.560693 8537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:11.560842 8537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:11.561852 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "93f616b7eb484062b6302ca7e361f36a"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "f94741726aed5a39e8283255afb739d1"
server_key_iv: "f71452c04849d67231898ef8a77d696e"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.562211 8537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "93f616b7eb484062b6302ca7e361f36a"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "f94741726aed5a39e8283255afb739d1"
server_key_iv: "f71452c04849d67231898ef8a77d696e"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.563618 8537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:11.564316 8552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.564476 8537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:11.564587 8537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "93f616b7eb484062b6302ca7e361f36a"
format_stamp: "Formatted at 2026-08-06 06:46:11 on dist-test-slave-5l3k"
server_key: "f94741726aed5a39e8283255afb739d1"
server_key_iv: "f71452c04849d67231898ef8a77d696e"
server_key_version: "encryptionkey@0"
I20260806 06:46:11.564670 8537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:11.588724 8537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:11.590603 8537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:11.590758 8537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:11.591315 8537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:11.591373 8537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.591410 8537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:11.591431 8537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.598631 8537 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:42465
I20260806 06:46:11.598714 8675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:42465 every 8 connection(s)
I20260806 06:46:11.599239 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8537
I20260806 06:46:11.599352 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:11.599584 6373 external_mini_cluster.cc:1428] Setting key d36d6b5840c77013c202187f859d13fb
I20260806 06:46:11.599017 8537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:11.601723 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.591759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:47741 (local address 127.6.57.126:37923)
0806 06:46:11.591839 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.591841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.592028 (+ 187us) server_negotiation.cc:407] Connection header received
0806 06:46:11.592489 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.592491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.592506 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.592582 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.593243 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.594067 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.595747 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.596069 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.596219 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.598569 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.598571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.598571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.598579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.600336 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.600796 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.600797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.600797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.600872 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.601160 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.601161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.601162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.601281 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.601348 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.601584 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.601660 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:46:11.601847 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.591715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.67:47741)
0806 06:46:11.591892 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.591931 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.592395 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.592658 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.592662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.592882 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.593145 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.593150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.594155 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.594157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.595650 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.595653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.596251 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.596252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.596390 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.596917 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.596933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.598440 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.600446 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.600451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.600453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.600727 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.600949 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.600950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.600950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.601068 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.601350 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.601354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.601463 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.601552 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.601670 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260806 06:46:11.602299 8676 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37923
I20260806 06:46:11.602402 8676 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:11.602602 8676 heartbeater.cc:511] Master 127.6.57.126:37923 requested a full tablet report, sending...
I20260806 06:46:11.602996 8172 ts_manager.cc:194] Registered new tserver with Master: 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:11.603468 8172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:47741
I20260806 06:46:11.608807 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.603384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34417 (local address 127.6.57.126:37923)
0806 06:46:11.603428 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.603430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.603580 (+ 150us) server_negotiation.cc:407] Connection header received
0806 06:46:11.603631 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.603632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.603642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.603672 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.604072 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.604526 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.605285 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.605487 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.605561 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.606347 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.606348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.606348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.606355 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.607650 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.608029 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.608030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.608031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.608086 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.608301 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.608302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.608303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.608412 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.608486 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.608655 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.608722 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:11.608840 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.603300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.6.57.67:34417)
0806 06:46:11.603495 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.603506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.603572 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.603733 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.603734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.603947 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.603994 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.603995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.604620 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.604621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.605202 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.605204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.605561 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.605562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.605639 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.605985 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.605989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.606257 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.607765 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.607766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.607766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.607940 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.608170 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.608171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.608171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.608236 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.608490 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.608491 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.608534 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.608620 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.608722 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260806 06:46:11.611411 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.635347 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.628416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.6.57.126:37923)
0806 06:46:11.628485 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.628487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.628495 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:11.628525 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.628526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.628535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.628598 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.629178 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.629670 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.630403 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.630651 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.630725 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.632847 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.632849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.632849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.632856 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.634127 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.634516 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.634517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.634518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.634573 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.634865 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.634865 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.634866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.634961 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.635032 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.635228 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.635286 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:46:11.637064 8172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:11.638743 8172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:11.645231 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:42465 (local address 127.0.0.1:51408)
0806 06:46:11.641580 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.641588 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.641622 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.641837 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.641841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.641843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.641909 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.641910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.642958 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.642959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.643968 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.643971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.644948 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.644949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.645021 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.645033 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.645091 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.645166 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260806 06:46:11.645268 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.6.57.67:42465)
0806 06:46:11.641649 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.641654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.641664 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:11.641701 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.641703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.641717 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.641814 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.642012 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.642868 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.644089 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.644852 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.644933 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.645005 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.645119 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.645165 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260806 06:46:11.646067 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:38531 (local address 127.0.0.1:37402)
0806 06:46:11.641346 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.641387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.641451 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.641904 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.641905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.641907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.641965 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.641966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.643090 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.643090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.643983 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.643986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.645797 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.645798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.645863 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.645868 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.645929 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.645971 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":51,"thread_start_us":59,"threads_started":1}
I20260806 06:46:11.646121 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.6.57.65:38531)
0806 06:46:11.641731 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.641735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.641746 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:11.641781 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.641782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.641817 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.641911 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.642028 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.643004 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.644981 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.645719 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.645784 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.645821 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.645964 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.646016 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":91,"threads_started":1}
I20260806 06:46:11.646026 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 789cb9c3b1564eb9814a9df6209723da (DEFAULT_TABLE table=test-table [id=35e8e77527d24e5b9080871ddd78b388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.646443 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789cb9c3b1564eb9814a9df6209723da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.646807 8315 tablet_service.cc:1467] Processing CreateTablet for tablet 789cb9c3b1564eb9814a9df6209723da (DEFAULT_TABLE table=test-table [id=35e8e77527d24e5b9080871ddd78b388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.647181 8315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789cb9c3b1564eb9814a9df6209723da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.649075 8699 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Bootstrap starting.
I20260806 06:46:11.650071 8699 tablet_bootstrap.cc:654] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.650200 8700 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Bootstrap starting.
I20260806 06:46:11.650437 8699 log.cc:824] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:11.650975 8700 tablet_bootstrap.cc:654] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.651288 8700 log.cc:824] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:11.652073 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:36803 (local address 127.0.0.1:41064)
0806 06:46:11.641463 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.641472 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.641510 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.642808 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.642809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.642811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.642873 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.642875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.646015 (+ 3140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.646017 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.650849 (+ 4832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.650853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.651798 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.651800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.651859 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.651863 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.651955 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.651998 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
I20260806 06:46:11.652092 8700 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: No bootstrap required, opened a new log
I20260806 06:46:11.652146 8699 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: No bootstrap required, opened a new log
I20260806 06:46:11.652168 8700 ts_tablet_manager.cc:1397] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:46:11.652186 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.641525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.6.57.66:36803)
0806 06:46:11.642613 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.642617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.642628 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:11.642669 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.642670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.642685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.642799 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.643940 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.645897 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.650954 (+ 5057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.651692 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.651796 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.651853 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.652022 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.652066 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":79,"threads_started":1}
I20260806 06:46:11.652197 8699 ts_tablet_manager.cc:1397] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:46:11.652966 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 789cb9c3b1564eb9814a9df6209723da (DEFAULT_TABLE table=test-table [id=35e8e77527d24e5b9080871ddd78b388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.653367 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 789cb9c3b1564eb9814a9df6209723da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.654153 8699 raft_consensus.cc:348] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.654330 8699 raft_consensus.cc:374] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.654366 8699 raft_consensus.cc:729] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.654287 8700 raft_consensus.cc:348] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.654421 8700 raft_consensus.cc:374] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.654452 8700 raft_consensus.cc:729] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8285860a6d3941d9b2c755ec470f0d9d, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.654527 8699 consensus_queue.cc:260] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.654594 8700 consensus_queue.cc:260] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.654844 8700 ts_tablet_manager.cc:1428] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:46:11.654844 8699 ts_tablet_manager.cc:1428] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:11.654970 8383 heartbeater.cc:503] Master 127.6.57.126:37923 was elected leader, sending a full tablet report...
I20260806 06:46:11.654987 8676 heartbeater.cc:503] Master 127.6.57.126:37923 was elected leader, sending a full tablet report...
I20260806 06:46:11.656944 8705 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Bootstrap starting.
I20260806 06:46:11.657590 8705 tablet_bootstrap.cc:654] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.657888 8705 log.cc:824] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:11.658654 8705 tablet_bootstrap.cc:492] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: No bootstrap required, opened a new log
I20260806 06:46:11.658733 8705 ts_tablet_manager.cc:1397] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:11.660027 8705 raft_consensus.cc:348] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.660169 8705 raft_consensus.cc:374] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.660218 8705 raft_consensus.cc:729] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.660329 8705 consensus_queue.cc:260] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.660578 8705 ts_tablet_manager.cc:1428] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:11.660634 8528 heartbeater.cc:503] Master 127.6.57.126:37923 was elected leader, sending a full tablet report...
W20260806 06:46:11.681586 8529 tablet.cc:2369] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:11.711939 8708 raft_consensus.cc:482] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:11.712050 8708 raft_consensus.cc:504] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.712416 8708 leader_election.cc:288] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
W20260806 06:46:11.715703 8384 tablet.cc:2369] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:11.716315 8710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.712630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:38531 (local address 127.6.57.66:45293)
0806 06:46:11.712914 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.712926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.712987 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.713141 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.713142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.713143 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.713197 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.713198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.714046 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.714047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.715268 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.715271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.716063 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.716064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.716117 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.716120 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.716163 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.716192 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":94,"thread_start_us":128,"threads_started":1}
I20260806 06:46:11.716327 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.712734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:45293 (local address 127.6.57.65:38531)
0806 06:46:11.712877 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.712879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.712997 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:46:11.713043 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.713044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.713054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.713123 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.713266 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.713980 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.715365 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.715919 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.715989 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.716000 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.716202 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.716259 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260806 06:46:11.716394 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.712743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:55563 (local address 127.6.57.67:42465)
0806 06:46:11.712805 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.712807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.712817 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:11.712847 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.712848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.712861 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.712919 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.713126 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.713902 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.715365 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.715910 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.715989 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.716000 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.716270 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.716320 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:46:11.716410 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.712630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:42465 (local address 127.6.57.66:55563)
0806 06:46:11.712699 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.712715 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.712781 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.712948 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.712949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.712951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.713027 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.713028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.713982 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.713983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.715268 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.715271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.716078 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.716080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.716171 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.716177 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.716254 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.716292 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":47}
I20260806 06:46:11.716682 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789cb9c3b1564eb9814a9df6209723da" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d" is_pre_election: true
I20260806 06:46:11.716825 8337 raft_consensus.cc:2454] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:11.716784 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789cb9c3b1564eb9814a9df6209723da" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a" is_pre_election: true
I20260806 06:46:11.716965 8615 raft_consensus.cc:2454] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:11.717083 8411 leader_election.cc:302] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 8285860a6d3941d9b2c755ec470f0d9d; no voters:
I20260806 06:46:11.717263 8708 raft_consensus.cc:2781] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:11.717326 8708 raft_consensus.cc:482] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:11.717347 8708 raft_consensus.cc:3037] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.718446 8708 raft_consensus.cc:504] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.718639 8708 leader_election.cc:288] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Requested vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:11.718788 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789cb9c3b1564eb9814a9df6209723da" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a"
I20260806 06:46:11.718875 8615 raft_consensus.cc:3037] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.718868 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "789cb9c3b1564eb9814a9df6209723da" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d"
I20260806 06:46:11.718951 8337 raft_consensus.cc:3037] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.719676 8615 raft_consensus.cc:2454] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:11.719732 8337 raft_consensus.cc:2454] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:11.719897 8410 leader_election.cc:302] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:11.720055 8708 raft_consensus.cc:2781] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:11.720249 8708 raft_consensus.cc:686] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Becoming Leader. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Running, Role: LEADER
I20260806 06:46:11.720371 8708 consensus_queue.cc:237] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } }
I20260806 06:46:11.721258 8173 catalog_manager.cc:5495] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66). New cstate: current_term: 1 leader_uuid: "197fd40dcd134633a18ae59d054a9d3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } health_report { overall_health: HEALTHY } } }
Aug 06 06:46:11 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998771, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:11.766031 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.759264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.6.57.126:37923)
0806 06:46:11.759350 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.759352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.759488 (+ 136us) server_negotiation.cc:407] Connection header received
0806 06:46:11.759584 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.759584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.759596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.759675 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:11.760159 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.760641 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.761327 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.761510 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.761588 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.763605 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:11.763606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:11.763607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:11.763613 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:11.764779 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.765230 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.765232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.765232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.765290 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:11.765485 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:11.765486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:11.765486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:11.765595 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:11.765675 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.765830 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.765916 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 06:46:11.766112 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.759204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37923 (local address 127.0.0.1:38460)
0806 06:46:11.759403 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.759413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.759506 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.759681 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.759682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.760003 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:11.760050 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.760052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.760723 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.760724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.761240 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.761241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.761584 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.761584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.761649 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.762162 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:11.762167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:11.763504 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:11.764874 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.764909 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.764912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.765148 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.765356 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:11.765357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:11.765357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:11.765420 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:11.765664 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:11.765672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:11.765720 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.765805 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.765987 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":82,"thread_start_us":94,"threads_started":1}
I20260806 06:46:11.768321 8173 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:38460:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:46:11.768534 8173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:11.770197 8315 tablet_service.cc:1467] Processing CreateTablet for tablet b884c1cb550a40358a7204333891bd9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=33beb3408b734407b491adc627d97f04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:11.770200 8606 tablet_service.cc:1467] Processing CreateTablet for tablet b884c1cb550a40358a7204333891bd9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=33beb3408b734407b491adc627d97f04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:11.770221 8459 tablet_service.cc:1467] Processing CreateTablet for tablet b884c1cb550a40358a7204333891bd9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=33beb3408b734407b491adc627d97f04]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:11.770399 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b884c1cb550a40358a7204333891bd9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.770401 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b884c1cb550a40358a7204333891bd9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.770411 8315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b884c1cb550a40358a7204333891bd9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.771867 8700 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: Bootstrap starting.
I20260806 06:46:11.771931 8705 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: Bootstrap starting.
I20260806 06:46:11.771931 8699 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: Bootstrap starting.
I20260806 06:46:11.772282 8700 tablet_bootstrap.cc:654] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.772416 8705 tablet_bootstrap.cc:654] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.772594 8699 tablet_bootstrap.cc:654] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.773172 8705 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: No bootstrap required, opened a new log
I20260806 06:46:11.773226 8705 ts_tablet_manager.cc:1397] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:11.773370 8699 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: No bootstrap required, opened a new log
I20260806 06:46:11.773381 8700 tablet_bootstrap.cc:492] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: No bootstrap required, opened a new log
I20260806 06:46:11.773403 8699 ts_tablet_manager.cc:1397] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:11.773428 8700 ts_tablet_manager.cc:1397] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 06:46:11.773394 8705 raft_consensus.cc:348] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773504 8705 raft_consensus.cc:374] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.773533 8705 raft_consensus.cc:729] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.773516 8699 raft_consensus.cc:348] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773562 8699 raft_consensus.cc:374] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.773576 8699 raft_consensus.cc:729] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.773602 8705 consensus_queue.cc:260] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773597 8700 raft_consensus.cc:348] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773665 8700 raft_consensus.cc:374] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.773648 8699 consensus_queue.cc:260] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773690 8700 raft_consensus.cc:729] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8285860a6d3941d9b2c755ec470f0d9d, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.773710 8705 ts_tablet_manager.cc:1428] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.773731 8699 ts_tablet_manager.cc:1428] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.773739 8700 consensus_queue.cc:260] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.773746 8708 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.773826 8708 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.773891 8700 ts_tablet_manager.cc:1428] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.773957 8703 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.774046 8703 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.774087 8704 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.774173 8704 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.832742 8703 raft_consensus.cc:482] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:11.832837 8703 raft_consensus.cc:504] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.833189 8703 leader_election.cc:288] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803)
I20260806 06:46:11.836473 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.833562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:45183 (local address 127.6.57.66:36803)
0806 06:46:11.833616 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.833619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.833772 (+ 153us) server_negotiation.cc:407] Connection header received
0806 06:46:11.833844 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.833845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.833857 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.833925 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.834142 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.834670 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.835756 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.836104 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.836193 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.836205 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.836370 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.836406 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:11.836493 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.833418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:36803 (local address 127.6.57.67:45183)
0806 06:46:11.833677 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.833686 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.833781 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.833931 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.833932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.833934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.834025 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.834026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.834752 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.834754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.835663 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.835665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.836218 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.836219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.836288 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.836291 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.836353 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.836392 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":106,"thread_start_us":119,"threads_started":1}
I20260806 06:46:11.836674 8565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.833418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:38531 (local address 127.6.57.67:57091)
0806 06:46:11.833481 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.833496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.833551 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.833743 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.833744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.833747 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:11.833833 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.833835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.834772 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.834773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.835712 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.835715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.836337 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.836339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.836431 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.836434 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.836525 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.836573 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":2}
I20260806 06:46:11.836874 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.833515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:57091 (local address 127.6.57.65:38531)
0806 06:46:11.833562 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.833564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.833590 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:46:11.833618 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.833618 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.833629 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.833685 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:11.833965 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.834665 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.835825 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.836228 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.836324 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.836336 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.836718 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.836788 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:46:11.837030 8480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b884c1cb550a40358a7204333891bd9a" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197fd40dcd134633a18ae59d054a9d3e" is_pre_election: true
I20260806 06:46:11.837181 8480 raft_consensus.cc:2454] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 0.
I20260806 06:46:11.837445 8553 leader_election.cc:302] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:11.837602 8703 raft_consensus.cc:2781] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:11.837646 8703 raft_consensus.cc:482] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:11.837664 8703 raft_consensus.cc:3037] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.837663 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b884c1cb550a40358a7204333891bd9a" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d" is_pre_election: true
I20260806 06:46:11.837817 8337 raft_consensus.cc:2454] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 0.
I20260806 06:46:11.838414 8703 raft_consensus.cc:504] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.838584 8703 leader_election.cc:288] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 election: Requested vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803)
I20260806 06:46:11.838794 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b884c1cb550a40358a7204333891bd9a" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d"
I20260806 06:46:11.838794 8480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b884c1cb550a40358a7204333891bd9a" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197fd40dcd134633a18ae59d054a9d3e"
I20260806 06:46:11.838871 8337 raft_consensus.cc:3037] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.838871 8480 raft_consensus.cc:3037] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.839624 8480 raft_consensus.cc:2454] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 1.
I20260806 06:46:11.839632 8337 raft_consensus.cc:2454] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 1.
I20260806 06:46:11.839836 8553 leader_election.cc:302] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:11.839931 8703 raft_consensus.cc:2781] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:11.840083 8703 raft_consensus.cc:686] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [term 1 LEADER]: Becoming Leader. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Running, Role: LEADER
I20260806 06:46:11.840191 8703 consensus_queue.cc:237] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.840662 8724 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: TxnStatusTablet state changed. Reason: New leader 93f616b7eb484062b6302ca7e361f36a. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.840786 8724 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:11.840992 8726 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:46:11.841013 8172 catalog_manager.cc:5495] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a reported cstate change: term changed from 0 to 1, leader changed from <none> to 93f616b7eb484062b6302ca7e361f36a (127.6.57.67). New cstate: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } health_report { overall_health: HEALTHY } } }
I20260806 06:46:11.841120 8726 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:46:11.849381 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.846928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.6.57.67:42465)
0806 06:46:11.846993 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.846995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.847004 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:11.847033 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.847034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.847050 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.847107 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:11.847258 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.847734 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.848558 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.848787 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.848866 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.849058 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.849157 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.849243 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.849295 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.849316 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:46:11.849411 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.846814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:42465 (local address 127.0.0.1:51410)
0806 06:46:11.846888 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.846899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.846963 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.847127 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.847128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.847129 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:11.847175 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.847177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.847818 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.847819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.848473 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.848476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.848866 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.848867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.848935 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.848958 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.849223 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.849233 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.849289 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.849334 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:46:11.850638 8677 tablet.cc:2369] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:11.851104 8337 raft_consensus.cc:1261] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Refusing update from remote peer 93f616b7eb484062b6302ca7e361f36a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:11.851104 8480 raft_consensus.cc:1261] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Refusing update from remote peer 93f616b7eb484062b6302ca7e361f36a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:11.851357 8724 consensus_queue.cc:1035] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:11.851490 8703 consensus_queue.cc:1035] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:11.852574 8708 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: TxnStatusTablet state changed. Reason: New leader 93f616b7eb484062b6302ca7e361f36a. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.852680 8708 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.852769 8704 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: TxnStatusTablet state changed. Reason: New leader 93f616b7eb484062b6302ca7e361f36a. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.852850 8704 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.854213 8703 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.854295 8703 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:11.854379 8703 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.854435 8703 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 93f616b7eb484062b6302ca7e361f36a: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:11.854673 8729 mvcc.cc:205] Tried to move back new op lower bound from 7315450969499791360 to 7315450969457995776. Current Snapshot: MvccSnapshot[applied={T|T < 7315450969499791360 or (T in {7315450969499791360})}]
I20260806 06:46:11.854856 8731 mvcc.cc:205] Tried to move back new op lower bound from 7315450969499791360 to 7315450969457995776. Current Snapshot: MvccSnapshot[applied={T|T < 7315450969499791360}]
I20260806 06:46:11.854941 8708 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.855041 8708 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 197fd40dcd134633a18ae59d054a9d3e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.856096 8704 tablet_replica.cc:439] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } } }
I20260806 06:46:11.856213 8704 tablet_replica.cc:442] T b884c1cb550a40358a7204333891bd9a P 8285860a6d3941d9b2c755ec470f0d9d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:11.859164 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.856294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.6.57.66:36803)
0806 06:46:11.856338 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.856341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.856349 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:11.856380 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.856381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.856402 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.856472 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:11.856656 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.857180 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.858401 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.858642 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.858694 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.858871 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.858947 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.858995 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.859079 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.859094 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:11.863755 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.861588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:42227 (local address 127.6.57.67:42465)
0806 06:46:11.861645 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.861647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.861656 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:11.861678 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.861679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.861689 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.861730 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:11.861861 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.862273 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.863067 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.863254 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.863338 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.863520 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.863557 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.863625 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.863678 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.863696 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:46:11.863786 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.861511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:42465 (local address 127.6.57.66:42227)
0806 06:46:11.861585 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.861595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.861634 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.861741 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.861741 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.861742 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:11.861797 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.861798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.862361 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.862362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.862981 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.862983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.863341 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.863343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.863406 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.863427 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.863627 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.863629 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.863677 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.863715 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:46:11.865801 8337 raft_consensus.cc:1261] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:11.865991 8615 raft_consensus.cc:1261] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:11.866283 8711 consensus_queue.cc:1035] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:11.866411 8708 consensus_queue.cc:1035] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:11.875862 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.873228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:51411 (local address 127.6.57.66:36803)
0806 06:46:11.873261 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:11.873263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:11.873333 (+ 70us) server_negotiation.cc:407] Connection header received
0806 06:46:11.873394 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:11.873394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:11.873405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:11.873457 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:11.873580 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.874011 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.875124 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.875319 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.875401 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.875630 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.875660 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.875722 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:11.875784 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:46:11.875800 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:46:11.875888 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:11.873172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:36803 (local address 127.6.57.67:51411)
0806 06:46:11.873289 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:11.873299 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:11.873347 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:11.873461 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:11.873462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:11.873462 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:11.873514 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.873515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.874093 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.874094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.875050 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:11.875053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:11.875403 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:11.875404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:11.875477 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:11.875492 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:11.875734 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:11.875737 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:11.875781 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:46:11.875820 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 06:46:11.885434 8173 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:11.885572 8173 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:11.886385 8173 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Sending DeleteTablet for 3 replicas of tablet 789cb9c3b1564eb9814a9df6209723da
I20260806 06:46:11.886899 8315 tablet_service.cc:1514] Processing DeleteTablet for tablet 789cb9c3b1564eb9814a9df6209723da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:11 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37402
I20260806 06:46:11.887130 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 789cb9c3b1564eb9814a9df6209723da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:11 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41064
I20260806 06:46:11.887368 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 789cb9c3b1564eb9814a9df6209723da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:11 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:51408
I20260806 06:46:11.887427 8746 tablet_replica.cc:330] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: stopping tablet replica
I20260806 06:46:11.887567 8173 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:46:11.887633 8746 raft_consensus.cc:2229] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 06:46:11.887717 8173 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:11.887902 8746 raft_consensus.cc:2258] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:11.888388 8746 ts_tablet_manager.cc:1905] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:11.889371 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf279e7e32d4d95812380013f5928e8 (DEFAULT_TABLE table=test-table [id=4dc39fb344994de3a715a792a5fca865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.889513 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf279e7e32d4d95812380013f5928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.890328 8746 ts_tablet_manager.cc:1918] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:11.890463 8746 log.cc:1196] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/789cb9c3b1564eb9814a9df6209723da
I20260806 06:46:11.890569 8315 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf279e7e32d4d95812380013f5928e8 (DEFAULT_TABLE table=test-table [id=4dc39fb344994de3a715a792a5fca865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.890772 8315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf279e7e32d4d95812380013f5928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.891022 8746 ts_tablet_manager.cc:1939] T 789cb9c3b1564eb9814a9df6209723da P 8285860a6d3941d9b2c755ec470f0d9d: Deleting consensus metadata
I20260806 06:46:11.892016 8700 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Bootstrap starting.
I20260806 06:46:11.892112 8160 catalog_manager.cc:4841] TS 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531): tablet 789cb9c3b1564eb9814a9df6209723da (table test-table [id=35e8e77527d24e5b9080871ddd78b388]) successfully deleted
I20260806 06:46:11.892416 8700 tablet_bootstrap.cc:654] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.892402 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 6bf279e7e32d4d95812380013f5928e8 (DEFAULT_TABLE table=test-table [id=4dc39fb344994de3a715a792a5fca865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:11.892542 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bf279e7e32d4d95812380013f5928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:11.893960 8699 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Bootstrap starting.
I20260806 06:46:11.894431 8699 tablet_bootstrap.cc:654] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.895457 8705 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Bootstrap starting.
I20260806 06:46:11.895707 8699 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: No bootstrap required, opened a new log
I20260806 06:46:11.895751 8699 ts_tablet_manager.cc:1397] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:11.895895 8705 tablet_bootstrap.cc:654] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:11.895906 8699 raft_consensus.cc:348] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.895970 8699 raft_consensus.cc:374] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.895994 8699 raft_consensus.cc:729] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.896055 8699 consensus_queue.cc:260] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.896158 8699 ts_tablet_manager.cc:1428] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:11.896687 8747 tablet_replica.cc:330] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: stopping tablet replica
I20260806 06:46:11.896802 8747 raft_consensus.cc:2229] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:11.896917 8747 raft_consensus.cc:2258] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:11.897091 8748 tablet_replica.cc:330] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: stopping tablet replica
I20260806 06:46:11.897194 8748 raft_consensus.cc:2229] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:11.897254 8748 raft_consensus.cc:2258] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:11.897452 8747 ts_tablet_manager.cc:1905] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:11.897655 8748 ts_tablet_manager.cc:1905] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:11.899152 8747 ts_tablet_manager.cc:1918] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:11.899274 8747 log.cc:1196] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/789cb9c3b1564eb9814a9df6209723da
I20260806 06:46:11.899572 8747 ts_tablet_manager.cc:1939] T 789cb9c3b1564eb9814a9df6209723da P 197fd40dcd134633a18ae59d054a9d3e: Deleting consensus metadata
I20260806 06:46:11.899756 8748 ts_tablet_manager.cc:1918] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:11.900096 8157 catalog_manager.cc:4841] TS 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803): tablet 789cb9c3b1564eb9814a9df6209723da (table test-table [id=35e8e77527d24e5b9080871ddd78b388]) successfully deleted
I20260806 06:46:11.900300 8748 log.cc:1196] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/789cb9c3b1564eb9814a9df6209723da
I20260806 06:46:11.900606 8748 ts_tablet_manager.cc:1939] T 789cb9c3b1564eb9814a9df6209723da P 93f616b7eb484062b6302ca7e361f36a: Deleting consensus metadata
I20260806 06:46:11.901026 8159 catalog_manager.cc:4841] TS 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465): tablet 789cb9c3b1564eb9814a9df6209723da (table test-table [id=35e8e77527d24e5b9080871ddd78b388]) successfully deleted
I20260806 06:46:11.902845 8705 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: No bootstrap required, opened a new log
I20260806 06:46:11.902892 8705 ts_tablet_manager.cc:1397] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 06:46:11.903038 8705 raft_consensus.cc:348] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.903115 8705 raft_consensus.cc:374] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.903142 8705 raft_consensus.cc:729] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.903208 8705 consensus_queue.cc:260] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.903293 8705 ts_tablet_manager.cc:1428] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.906912 8700 tablet_bootstrap.cc:492] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: No bootstrap required, opened a new log
I20260806 06:46:11.906975 8700 ts_tablet_manager.cc:1397] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.000s
I20260806 06:46:11.907114 8700 raft_consensus.cc:348] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.907187 8700 raft_consensus.cc:374] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:11.907208 8700 raft_consensus.cc:729] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8285860a6d3941d9b2c755ec470f0d9d, State: Initialized, Role: FOLLOWER
I20260806 06:46:11.907255 8700 consensus_queue.cc:260] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.907356 8700 ts_tablet_manager.cc:1428] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:11.985647 8711 raft_consensus.cc:482] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:11.985771 8711 raft_consensus.cc:504] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.985955 8711 leader_election.cc:288] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:11.986258 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf279e7e32d4d95812380013f5928e8" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d" is_pre_election: true
I20260806 06:46:11.986289 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf279e7e32d4d95812380013f5928e8" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a" is_pre_election: true
I20260806 06:46:11.986368 8337 raft_consensus.cc:2454] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:11.986402 8615 raft_consensus.cc:2454] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:11.986588 8411 leader_election.cc:302] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 8285860a6d3941d9b2c755ec470f0d9d; no voters:
I20260806 06:46:11.986730 8711 raft_consensus.cc:2781] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:11.986786 8711 raft_consensus.cc:482] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:11.986814 8711 raft_consensus.cc:3037] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.987787 8711 raft_consensus.cc:504] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.987939 8711 leader_election.cc:288] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Requested vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:11.988145 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf279e7e32d4d95812380013f5928e8" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a"
I20260806 06:46:11.988216 8615 raft_consensus.cc:3037] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.988214 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bf279e7e32d4d95812380013f5928e8" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d"
I20260806 06:46:11.988289 8337 raft_consensus.cc:3037] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:11.989004 8615 raft_consensus.cc:2454] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:11.989140 8337 raft_consensus.cc:2454] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:11.989189 8410 leader_election.cc:302] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:11.989264 8711 raft_consensus.cc:2781] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:11.989298 8711 raft_consensus.cc:686] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Becoming Leader. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Running, Role: LEADER
I20260806 06:46:11.989347 8711 consensus_queue.cc:237] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:11.990026 8171 catalog_manager.cc:5495] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66). New cstate: current_term: 1 leader_uuid: "197fd40dcd134633a18ae59d054a9d3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:12.019287 8337 raft_consensus.cc:1261] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:12.019335 8615 raft_consensus.cc:1261] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:12.019513 8711 consensus_queue.cc:1035] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:12.019615 8708 consensus_queue.cc:1035] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:12.021384 8730 mvcc.cc:205] Tried to move back new op lower bound from 7315450970189611008 to 7315450970068729856. Current Snapshot: MvccSnapshot[applied={T|T < 7315450970189611008}]
W20260806 06:46:12.024394 8731 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:46:13.542941 8171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:13.543044 8171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:13.543905 8171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Sending DeleteTablet for 3 replicas of tablet 6bf279e7e32d4d95812380013f5928e8
I20260806 06:46:13.544168 8315 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf279e7e32d4d95812380013f5928e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37402
I20260806 06:46:13.544226 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf279e7e32d4d95812380013f5928e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41064
I20260806 06:46:13.544274 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 6bf279e7e32d4d95812380013f5928e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:51408
I20260806 06:46:13.545912 8781 tablet_replica.cc:330] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: stopping tablet replica
I20260806 06:46:13.546013 8781 raft_consensus.cc:2229] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:13.546088 8781 raft_consensus.cc:2258] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.546360 8781 ts_tablet_manager.cc:1905] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:13.546568 8782 tablet_replica.cc:330] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: stopping tablet replica
I20260806 06:46:13.546672 8782 raft_consensus.cc:2229] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:13.546754 8782 raft_consensus.cc:2258] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.547013 8782 ts_tablet_manager.cc:1905] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:13.547489 8780 tablet_replica.cc:330] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: stopping tablet replica
I20260806 06:46:13.547561 8780 raft_consensus.cc:2229] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:13.547592 8780 raft_consensus.cc:2258] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.547821 8780 ts_tablet_manager.cc:1905] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:13.548133 8781 ts_tablet_manager.cc:1918] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:46:13.548200 8781 log.cc:1196] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/6bf279e7e32d4d95812380013f5928e8
I20260806 06:46:13.548461 8781 ts_tablet_manager.cc:1939] T 6bf279e7e32d4d95812380013f5928e8 P 197fd40dcd134633a18ae59d054a9d3e: Deleting consensus metadata
I20260806 06:46:13.548835 8157 catalog_manager.cc:4841] TS 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803): tablet 6bf279e7e32d4d95812380013f5928e8 (table test-table [id=4dc39fb344994de3a715a792a5fca865]) successfully deleted
I20260806 06:46:13.549083 8780 ts_tablet_manager.cc:1918] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:46:13.549141 8780 log.cc:1196] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/6bf279e7e32d4d95812380013f5928e8
I20260806 06:46:13.549293 8780 ts_tablet_manager.cc:1939] T 6bf279e7e32d4d95812380013f5928e8 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting consensus metadata
I20260806 06:46:13.549479 8782 ts_tablet_manager.cc:1918] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:46:13.549568 8782 log.cc:1196] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/6bf279e7e32d4d95812380013f5928e8
I20260806 06:46:13.549818 8782 ts_tablet_manager.cc:1939] T 6bf279e7e32d4d95812380013f5928e8 P 93f616b7eb484062b6302ca7e361f36a: Deleting consensus metadata
I20260806 06:46:13.549841 8160 catalog_manager.cc:4841] TS 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531): tablet 6bf279e7e32d4d95812380013f5928e8 (table test-table [id=4dc39fb344994de3a715a792a5fca865]) successfully deleted
I20260806 06:46:13.550100 8159 catalog_manager.cc:4841] TS 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465): tablet 6bf279e7e32d4d95812380013f5928e8 (table test-table [id=4dc39fb344994de3a715a792a5fca865]) successfully deleted
Aug 06 06:46:13 dist-test-slave-5l3k krb5kdc[8127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998773, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:46:13 dist-test-slave-5l3k krb5kdc[8127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998773, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:13.570900 8792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:13.563454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.6.57.126:37923)
0806 06:46:13.563623 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:13.563625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:13.563636 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:13.563662 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:13.563662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:13.563674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:13.563738 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:13.564346 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.564866 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.565607 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.565796 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.565844 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:13.568394 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:13.568396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:13.568396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:13.568402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:13.569781 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:13.570139 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:13.570140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:13.570140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:13.570192 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:13.570387 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:13.570387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:13.570387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:13.570539 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:13.570628 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:13.570750 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:46:13.570817 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260806 06:46:13.572374 8171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:13.572556 8171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:13.573935 8315 tablet_service.cc:1467] Processing CreateTablet for tablet 5debe7d21d47461895bbcf6db107f855 (DEFAULT_TABLE table=test-table [id=a40b8834f8df4240bdee1e57d3642ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.573979 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 5debe7d21d47461895bbcf6db107f855 (DEFAULT_TABLE table=test-table [id=a40b8834f8df4240bdee1e57d3642ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.574097 8315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5debe7d21d47461895bbcf6db107f855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.574152 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5debe7d21d47461895bbcf6db107f855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.575340 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 5debe7d21d47461895bbcf6db107f855 (DEFAULT_TABLE table=test-table [id=a40b8834f8df4240bdee1e57d3642ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.575369 8793 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Bootstrap starting.
I20260806 06:46:13.575382 8794 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Bootstrap starting.
I20260806 06:46:13.575497 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5debe7d21d47461895bbcf6db107f855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.575742 8793 tablet_bootstrap.cc:654] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.575830 8794 tablet_bootstrap.cc:654] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.576362 8793 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: No bootstrap required, opened a new log
I20260806 06:46:13.576427 8793 ts_tablet_manager.cc:1397] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:13.576553 8793 raft_consensus.cc:348] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.576630 8793 raft_consensus.cc:374] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.576664 8793 raft_consensus.cc:729] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.576718 8793 consensus_queue.cc:260] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.576840 8793 ts_tablet_manager.cc:1428] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:13.576915 8797 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Bootstrap starting.
I20260806 06:46:13.577319 8797 tablet_bootstrap.cc:654] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.577584 8794 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: No bootstrap required, opened a new log
I20260806 06:46:13.577667 8794 ts_tablet_manager.cc:1397] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:13.577806 8794 raft_consensus.cc:348] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.577883 8794 raft_consensus.cc:374] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.577906 8794 raft_consensus.cc:729] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8285860a6d3941d9b2c755ec470f0d9d, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.577936 8797 tablet_bootstrap.cc:492] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: No bootstrap required, opened a new log
I20260806 06:46:13.577957 8794 consensus_queue.cc:260] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.577991 8797 ts_tablet_manager.cc:1397] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:13.578114 8797 raft_consensus.cc:348] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.578145 8794 ts_tablet_manager.cc:1428] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:13.578187 8797 raft_consensus.cc:374] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.578208 8797 raft_consensus.cc:729] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.578253 8797 consensus_queue.cc:260] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.578397 8797 ts_tablet_manager.cc:1428] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:13.620632 8760 raft_consensus.cc:482] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:13.620748 8760 raft_consensus.cc:504] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.621021 8760 leader_election.cc:288] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803)
I20260806 06:46:13.621263 8480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5debe7d21d47461895bbcf6db107f855" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197fd40dcd134633a18ae59d054a9d3e" is_pre_election: true
I20260806 06:46:13.621246 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5debe7d21d47461895bbcf6db107f855" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d" is_pre_election: true
I20260806 06:46:13.621359 8480 raft_consensus.cc:2454] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 0.
I20260806 06:46:13.621359 8337 raft_consensus.cc:2454] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 0.
I20260806 06:46:13.621523 8556 leader_election.cc:302] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8285860a6d3941d9b2c755ec470f0d9d, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:13.621670 8760 raft_consensus.cc:2781] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:13.621714 8760 raft_consensus.cc:482] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:13.621733 8760 raft_consensus.cc:3037] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.622445 8760 raft_consensus.cc:504] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.622651 8760 leader_election.cc:288] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 election: Requested vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803)
I20260806 06:46:13.622798 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5debe7d21d47461895bbcf6db107f855" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d"
I20260806 06:46:13.622824 8480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5debe7d21d47461895bbcf6db107f855" candidate_uuid: "93f616b7eb484062b6302ca7e361f36a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "197fd40dcd134633a18ae59d054a9d3e"
I20260806 06:46:13.622891 8337 raft_consensus.cc:3037] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.622934 8480 raft_consensus.cc:3037] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.623499 8337 raft_consensus.cc:2454] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 1.
I20260806 06:46:13.623683 8480 raft_consensus.cc:2454] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93f616b7eb484062b6302ca7e361f36a in term 1.
I20260806 06:46:13.623693 8556 leader_election.cc:302] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8285860a6d3941d9b2c755ec470f0d9d, 93f616b7eb484062b6302ca7e361f36a; no voters:
I20260806 06:46:13.623790 8760 raft_consensus.cc:2781] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:13.623836 8760 raft_consensus.cc:686] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 1 LEADER]: Becoming Leader. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Running, Role: LEADER
I20260806 06:46:13.623893 8760 consensus_queue.cc:237] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.624518 8173 catalog_manager.cc:5495] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a reported cstate change: term changed from 0 to 1, leader changed from <none> to 93f616b7eb484062b6302ca7e361f36a (127.6.57.67). New cstate: current_term: 1 leader_uuid: "93f616b7eb484062b6302ca7e361f36a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } health_report { overall_health: HEALTHY } } }
I20260806 06:46:13.640286 8802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:13.637792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.6.57.67:42465)
0806 06:46:13.637943 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:13.637945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:13.637953 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:13.637982 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:13.637982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:13.637992 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:13.638054 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:13.638172 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.638731 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.639579 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.639766 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.639816 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:13.639942 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:13.639969 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:13.640009 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:13.640156 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:46:13.640175 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":102,"threads_started":1}
I20260806 06:46:13.644649 8802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:13.642373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:42787 (local address 127.6.57.67:42465)
0806 06:46:13.642416 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:13.642417 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:13.642557 (+ 140us) server_negotiation.cc:407] Connection header received
0806 06:46:13.642614 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:13.642614 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:13.642624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:13.642669 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:13.642741 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.643216 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.643919 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.644172 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.644227 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:13.644413 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:13.644452 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:13.644500 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:13.644552 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:46:13.644570 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:13.644661 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:13.642301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:42465 (local address 127.6.57.67:42787)
0806 06:46:13.642462 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:13.642470 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:13.642574 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:13.642672 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:13.642673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:13.642674 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:13.642708 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:13.642709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.643283 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.643284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:13.643833 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:13.643835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.644245 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.644246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:13.644286 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:13.644289 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:13.644515 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:13.644517 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:13.644549 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:46:13.644583 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260806 06:46:13.646797 8480 raft_consensus.cc:1261] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Refusing update from remote peer 93f616b7eb484062b6302ca7e361f36a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:13.646806 8337 raft_consensus.cc:1261] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Refusing update from remote peer 93f616b7eb484062b6302ca7e361f36a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:13.646991 8760 consensus_queue.cc:1035] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:13.647131 8760 consensus_queue.cc:1035] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [LEADER]: Connected to new peer: Peer: permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:13.658149 8171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:13.658265 8171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:13.658941 8171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Sending DeleteTablet for 3 replicas of tablet 5debe7d21d47461895bbcf6db107f855
I20260806 06:46:13.659210 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 5debe7d21d47461895bbcf6db107f855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41064
I20260806 06:46:13.659217 8315 tablet_service.cc:1514] Processing DeleteTablet for tablet 5debe7d21d47461895bbcf6db107f855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37402
I20260806 06:46:13.659310 8781 tablet_replica.cc:330] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: stopping tablet replica
I20260806 06:46:13.659363 8781 raft_consensus.cc:2229] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:13.659363 8780 tablet_replica.cc:330] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: stopping tablet replica
I20260806 06:46:13.659360 8606 tablet_service.cc:1514] Processing DeleteTablet for tablet 5debe7d21d47461895bbcf6db107f855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:13 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:51408
I20260806 06:46:13.659401 8781 raft_consensus.cc:2258] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.659420 8780 raft_consensus.cc:2229] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:13.659461 8780 raft_consensus.cc:2258] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.659454 8782 tablet_replica.cc:330] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: stopping tablet replica
I20260806 06:46:13.659529 8782 raft_consensus.cc:2229] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:13.659622 8782 raft_consensus.cc:2258] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:13.659825 8171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 06:46:13.659953 8782 ts_tablet_manager.cc:1905] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:46:13.659982 8171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:13.659983 8780 ts_tablet_manager.cc:1905] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:13.660063 8781 ts_tablet_manager.cc:1905] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:13.661128 8459 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3457c22c3f4a92914e6503af257311 (DEFAULT_TABLE table=test-table [id=92903c12d9bb4ba799b334452dbf9077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.661223 8459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3457c22c3f4a92914e6503af257311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.661684 8780 ts_tablet_manager.cc:1918] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:13.661723 8781 ts_tablet_manager.cc:1918] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:13.661752 8780 log.cc:1196] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/5debe7d21d47461895bbcf6db107f855
I20260806 06:46:13.661798 8781 log.cc:1196] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/5debe7d21d47461895bbcf6db107f855
I20260806 06:46:13.662060 8781 ts_tablet_manager.cc:1939] T 5debe7d21d47461895bbcf6db107f855 P 197fd40dcd134633a18ae59d054a9d3e: Deleting consensus metadata
I20260806 06:46:13.662065 8780 ts_tablet_manager.cc:1939] T 5debe7d21d47461895bbcf6db107f855 P 8285860a6d3941d9b2c755ec470f0d9d: Deleting consensus metadata
I20260806 06:46:13.662451 8782 ts_tablet_manager.cc:1918] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:13.662533 8157 catalog_manager.cc:4841] TS 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803): tablet 5debe7d21d47461895bbcf6db107f855 (table test-table [id=a40b8834f8df4240bdee1e57d3642ea7]) successfully deleted
I20260806 06:46:13.662554 8782 log.cc:1196] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/5debe7d21d47461895bbcf6db107f855
I20260806 06:46:13.662566 8160 catalog_manager.cc:4841] TS 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531): tablet 5debe7d21d47461895bbcf6db107f855 (table test-table [id=a40b8834f8df4240bdee1e57d3642ea7]) successfully deleted
I20260806 06:46:13.662707 8315 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3457c22c3f4a92914e6503af257311 (DEFAULT_TABLE table=test-table [id=92903c12d9bb4ba799b334452dbf9077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.662814 8315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3457c22c3f4a92914e6503af257311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.662853 8793 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Bootstrap starting.
I20260806 06:46:13.662941 8782 ts_tablet_manager.cc:1939] T 5debe7d21d47461895bbcf6db107f855 P 93f616b7eb484062b6302ca7e361f36a: Deleting consensus metadata
I20260806 06:46:13.663153 8793 tablet_bootstrap.cc:654] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.663751 8793 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: No bootstrap required, opened a new log
I20260806 06:46:13.663805 8793 ts_tablet_manager.cc:1397] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:13.663929 8793 raft_consensus.cc:348] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.664009 8793 raft_consensus.cc:374] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.664044 8793 raft_consensus.cc:729] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.664119 8793 consensus_queue.cc:260] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.664186 8606 tablet_service.cc:1467] Processing CreateTablet for tablet 7a3457c22c3f4a92914e6503af257311 (DEFAULT_TABLE table=test-table [id=92903c12d9bb4ba799b334452dbf9077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:13.664289 8606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a3457c22c3f4a92914e6503af257311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:13.664364 8793 ts_tablet_manager.cc:1428] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:13.664467 8159 catalog_manager.cc:4841] TS 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465): tablet 5debe7d21d47461895bbcf6db107f855 (table test-table [id=a40b8834f8df4240bdee1e57d3642ea7]) successfully deleted
I20260806 06:46:13.664613 8794 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d: Bootstrap starting.
I20260806 06:46:13.665026 8794 tablet_bootstrap.cc:654] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.665469 8797 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Bootstrap starting.
I20260806 06:46:13.665786 8797 tablet_bootstrap.cc:654] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:13.665879 8794 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d: No bootstrap required, opened a new log
I20260806 06:46:13.665911 8794 ts_tablet_manager.cc:1397] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:13.666062 8794 raft_consensus.cc:348] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.666147 8794 raft_consensus.cc:374] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.666177 8794 raft_consensus.cc:729] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8285860a6d3941d9b2c755ec470f0d9d, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.666216 8794 consensus_queue.cc:260] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.666313 8794 ts_tablet_manager.cc:1428] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:13.666952 8797 tablet_bootstrap.cc:492] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: No bootstrap required, opened a new log
I20260806 06:46:13.667001 8797 ts_tablet_manager.cc:1397] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:13.667093 8797 raft_consensus.cc:348] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.667151 8797 raft_consensus.cc:374] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:13.667171 8797 raft_consensus.cc:729] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93f616b7eb484062b6302ca7e361f36a, State: Initialized, Role: FOLLOWER
I20260806 06:46:13.667204 8797 consensus_queue.cc:260] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.667291 8797 ts_tablet_manager.cc:1428] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:13.925000 8774 raft_consensus.cc:482] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:13.925127 8774 raft_consensus.cc:504] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.925336 8774 leader_election.cc:288] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:13.925602 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3457c22c3f4a92914e6503af257311" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d" is_pre_election: true
I20260806 06:46:13.925602 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3457c22c3f4a92914e6503af257311" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a" is_pre_election: true
I20260806 06:46:13.925720 8615 raft_consensus.cc:2454] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:13.925720 8337 raft_consensus.cc:2454] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 0.
I20260806 06:46:13.925913 8411 leader_election.cc:302] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 8285860a6d3941d9b2c755ec470f0d9d; no voters:
I20260806 06:46:13.926092 8774 raft_consensus.cc:2781] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:13.926149 8774 raft_consensus.cc:482] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:13.926164 8774 raft_consensus.cc:3037] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.926939 8774 raft_consensus.cc:504] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.927076 8774 leader_election.cc:288] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Requested vote from peers 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531), 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465)
I20260806 06:46:13.927237 8337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3457c22c3f4a92914e6503af257311" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8285860a6d3941d9b2c755ec470f0d9d"
I20260806 06:46:13.927314 8337 raft_consensus.cc:3037] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.927277 8615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a3457c22c3f4a92914e6503af257311" candidate_uuid: "197fd40dcd134633a18ae59d054a9d3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93f616b7eb484062b6302ca7e361f36a"
I20260806 06:46:13.927388 8615 raft_consensus.cc:3037] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:13.927927 8337 raft_consensus.cc:2454] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:13.928092 8411 leader_election.cc:302] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 197fd40dcd134633a18ae59d054a9d3e, 8285860a6d3941d9b2c755ec470f0d9d; no voters:
I20260806 06:46:13.928200 8774 raft_consensus.cc:2781] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:13.928192 8615 raft_consensus.cc:2454] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 197fd40dcd134633a18ae59d054a9d3e in term 1.
I20260806 06:46:13.928272 8774 raft_consensus.cc:686] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Becoming Leader. State: Replica: 197fd40dcd134633a18ae59d054a9d3e, State: Running, Role: LEADER
I20260806 06:46:13.928326 8774 consensus_queue.cc:237] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } }
I20260806 06:46:13.928941 8173 catalog_manager.cc:5495] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66). New cstate: current_term: 1 leader_uuid: "197fd40dcd134633a18ae59d054a9d3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "197fd40dcd134633a18ae59d054a9d3e" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:13.952667 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:13.950602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.6.57.66:36803)
0806 06:46:13.950760 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:13.950780 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:13.950788 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:13.950807 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:13.950808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:13.950817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:13.950886 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:13.950988 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.951438 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.952099 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:13.952284 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:13.952336 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:13.952437 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:13.952461 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:13.952494 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:13.952549 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 06:46:13.952565 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260806 06:46:13.954741 8615 raft_consensus.cc:1261] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:13.954759 8337 raft_consensus.cc:1261] T 7a3457c22c3f4a92914e6503af257311 P 8285860a6d3941d9b2c755ec470f0d9d [term 1 FOLLOWER]: Refusing update from remote peer 197fd40dcd134633a18ae59d054a9d3e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:13.954957 8774 consensus_queue.cc:1035] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93f616b7eb484062b6302ca7e361f36a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 42465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:13.955080 8770 consensus_queue.cc:1035] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8285860a6d3941d9b2c755ec470f0d9d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 38531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:46:13.959509 8761 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 06:46:15.481742 8172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:15.481860 8172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:15.482822 8172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa62337afa541b2bf494f1b42da5d57: Sending DeleteTablet for 3 replicas of tablet 7a3457c22c3f4a92914e6503af257311
I20260806 06:46:15.483171 8315 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3457c22c3f4a92914e6503af257311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:15 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37402
I20260806 06:46:15.483212 8459 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3457c22c3f4a92914e6503af257311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:15 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41064
I20260806 06:46:15.483215 8605 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a3457c22c3f4a92914e6503af257311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:15 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:51408
I20260806 06:46:15.483319 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8248
I20260806 06:46:15.484380 8841 tablet_replica.cc:330] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: stopping tablet replica
I20260806 06:46:15.484438 8841 raft_consensus.cc:2229] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:15.484474 8841 raft_consensus.cc:2258] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:15.484586 8840 tablet_replica.cc:330] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: stopping tablet replica
I20260806 06:46:15.484638 8840 raft_consensus.cc:2229] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:15.484696 8840 raft_consensus.cc:2258] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:15.484915 8840 ts_tablet_manager.cc:1905] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:15.486140 8841 ts_tablet_manager.cc:1905] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:15.486398 8840 ts_tablet_manager.cc:1918] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:46:15.486527 8840 log.cc:1196] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/7a3457c22c3f4a92914e6503af257311
I20260806 06:46:15.486857 8840 ts_tablet_manager.cc:1939] T 7a3457c22c3f4a92914e6503af257311 P 197fd40dcd134633a18ae59d054a9d3e: Deleting consensus metadata
I20260806 06:46:15.487196 8157 catalog_manager.cc:4841] TS 197fd40dcd134633a18ae59d054a9d3e (127.6.57.66:36803): tablet 7a3457c22c3f4a92914e6503af257311 (table test-table [id=92903c12d9bb4ba799b334452dbf9077]) successfully deleted
I20260806 06:46:15.487922 8841 ts_tablet_manager.cc:1918] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 06:46:15.487995 8841 log.cc:1196] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/7a3457c22c3f4a92914e6503af257311
I20260806 06:46:15.488237 8841 ts_tablet_manager.cc:1939] T 7a3457c22c3f4a92914e6503af257311 P 93f616b7eb484062b6302ca7e361f36a: Deleting consensus metadata
I20260806 06:46:15.488556 8159 catalog_manager.cc:4841] TS 93f616b7eb484062b6302ca7e361f36a (127.6.57.67:42465): tablet 7a3457c22c3f4a92914e6503af257311 (table test-table [id=92903c12d9bb4ba799b334452dbf9077]) successfully deleted
W20260806 06:46:15.490517 8160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:38531 (error 108)
I20260806 06:46:15.491111 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:15.490910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:38531 (local address 127.0.0.1:37410)
0806 06:46:15.491035 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:15.491079 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:38531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260806 06:46:15.491142 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8392
W20260806 06:46:15.491221 8160 catalog_manager.cc:4584] TS 8285860a6d3941d9b2c755ec470f0d9d (127.6.57.65:38531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a3457c22c3f4a92914e6503af257311: Network error: Client connection negotiation failed: client connection to 127.6.57.65:38531: connect: Connection refused (error 111)
I20260806 06:46:15.499681 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8537
I20260806 06:46:15.506985 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8140
2026-08-06T06:46:15Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5777 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:15 dist-test-slave-5l3k krb5kdc[8847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:15 dist-test-slave-5l3k krb5kdc[8847](info): set up 2 sockets
Aug 06 06:46:15 dist-test-slave-5l3k krb5kdc[8847](info): commencing operation
krb5kdc: starting...
W20260806 06:46:16.560824 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:16 dist-test-slave-5l3k krb5kdc[8847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:16.679000 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41477
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:16.761770 8861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:16.761996 8861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:16.762017 8861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:16.763242 8861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:16.763301 8861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:16.763350 8861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:16.763378 8861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:16.763396 8861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:16.764935 8861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:16.765198 8861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:16.765444 8861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:46:16.770352 8861 server_base.cc:1034] running on GCE node
W20260806 06:46:16.770299 8866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:16.770325 8867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:16.770303 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:16.770725 8861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:16.770972 8861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:16.772176 8861 hybrid_clock.cc:648] HybridClock initialized: now 1785998776772159 us; error 34 us; skew 500 ppm
Aug 06 06:46:16 dist-test-slave-5l3k krb5kdc[8847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998776, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:16.774741 8861 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:16.775218 8861 webserver.cc:460] Webserver started at http://127.6.57.126:44495/ using document root <none> and password file <none>
I20260806 06:46:16.775467 8861 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:16.775514 8861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:16.775693 8861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:16.776736 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b3230e02dbca65b5d295e80e7f5519c6"
server_key_iv: "abd8763a55c274a3defcf0f4183a35d7"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.777112 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b3230e02dbca65b5d295e80e7f5519c6"
server_key_iv: "abd8763a55c274a3defcf0f4183a35d7"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.778534 8861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:16.779263 8876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:16.779433 8861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:16.779547 8861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b3230e02dbca65b5d295e80e7f5519c6"
server_key_iv: "abd8763a55c274a3defcf0f4183a35d7"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.779624 8861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:16.803838 8861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:16.805785 8861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:16.805987 8861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:16.810091 8861 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41477
I20260806 06:46:16.810142 8933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41477 every 8 connection(s)
I20260806 06:46:16.810659 8861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:16.811667 8934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:16.814060 8934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: Bootstrap starting.
I20260806 06:46:16.814102 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8861
I20260806 06:46:16.814184 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:16.814352 6373 external_mini_cluster.cc:1428] Setting key 99092428f1e04f9ff8bfc224557f33ec
I20260806 06:46:16.814851 8934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:16.815274 8934 log.cc:824] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:16.816020 8934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: No bootstrap required, opened a new log
I20260806 06:46:16.818408 8934 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } }
I20260806 06:46:16.818608 8934 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:16.818637 8934 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ba9ac1ade64e8b9f0fc67fea12ded0, State: Initialized, Role: FOLLOWER
I20260806 06:46:16.818794 8934 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } }
I20260806 06:46:16.818854 8934 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:16.818895 8934 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:16.818955 8934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:16.819612 8934 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } }
I20260806 06:46:16.819721 8934 leader_election.cc:302] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7ba9ac1ade64e8b9f0fc67fea12ded0; no voters:
I20260806 06:46:16.819904 8934 leader_election.cc:288] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:16.820214 8934 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:16.820225 8941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:16.821605 8941 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [term 1 LEADER]: Becoming Leader. State: Replica: a7ba9ac1ade64e8b9f0fc67fea12ded0, State: Running, Role: LEADER
I20260806 06:46:16.821739 8941 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } }
Aug 06 06:46:16 dist-test-slave-5l3k krb5kdc[8847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:16.822177 8953 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7ba9ac1ade64e8b9f0fc67fea12ded0. Latest consensus state: current_term: 1 leader_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } } }
I20260806 06:46:16.822264 8953 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:16.822755 8942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7ba9ac1ade64e8b9f0fc67fea12ded0" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41477 } } }
I20260806 06:46:16.822860 8942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0 [sys.catalog]: This master's current role is: LEADER
W20260806 06:46:16.822736 8955 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:16.823009 8955 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:16.823081 8956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:16.823236 8956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:16.824867 8956 catalog_manager.cc:1347] Generated new cluster ID: b1465d515efb43fbb0516cec900efc17
I20260806 06:46:16.824918 8956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:16.827054 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:16.814896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.6.57.126:41477)
0806 06:46:16.815136 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:16.815146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:16.815181 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:46:16.815732 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:16.815742 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:16.815805 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:16.815952 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:16.816624 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:16.818072 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:16.819106 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:16.819700 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:16.819855 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:16.823309 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:16.823313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:16.823314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:16.823326 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:16.825367 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:16.825797 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:16.825800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:16.825801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:16.825910 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:16.826124 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:16.826125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:16.826126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:16.826426 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:16.826624 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:16.826705 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:46:16.826842 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":91,"threads_started":1}
I20260806 06:46:16.841961 8956 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:16.842466 8956 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:16.853149 8956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7ba9ac1ade64e8b9f0fc67fea12ded0: Generated new TSK 0
I20260806 06:46:16.853348 8956 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:16.887799 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:16.968559 8962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:16.968744 8962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:16.968766 8962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:16.969870 8962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:16.969909 8962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:16.969935 8962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:16.971252 8962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:16.971421 8962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:16.971577 8962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:16.976333 8970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:16.976445 8967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:16.976456 8968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:16.976750 8962 server_base.cc:1034] running on GCE node
I20260806 06:46:16.976940 8962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:16.977144 8962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:16.978283 8962 hybrid_clock.cc:648] HybridClock initialized: now 1785998776978263 us; error 35 us; skew 500 ppm
Aug 06 06:46:16 dist-test-slave-5l3k krb5kdc[8847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998776, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:16.980779 8962 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:16.981168 8962 webserver.cc:460] Webserver started at http://127.6.57.65:44811/ using document root <none> and password file <none>
I20260806 06:46:16.981341 8962 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:16.981395 8962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:16.981478 8962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:16.982542 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "0e38e7f6dc4f4d4c8ebc0218ef6b2aa4"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b00ad95b71bbaaafd59b2ad4bdc4e686"
server_key_iv: "1b7a7af7f0cb12517b025130de8a1b2e"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.982844 8962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "0e38e7f6dc4f4d4c8ebc0218ef6b2aa4"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b00ad95b71bbaaafd59b2ad4bdc4e686"
server_key_iv: "1b7a7af7f0cb12517b025130de8a1b2e"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.984143 8962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:16.984824 8977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:16.984995 8962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:16.985073 8962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "0e38e7f6dc4f4d4c8ebc0218ef6b2aa4"
format_stamp: "Formatted at 2026-08-06 06:46:16 on dist-test-slave-5l3k"
server_key: "b00ad95b71bbaaafd59b2ad4bdc4e686"
server_key_iv: "1b7a7af7f0cb12517b025130de8a1b2e"
server_key_version: "encryptionkey@0"
I20260806 06:46:16.985124 8962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:16.995327 8962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:16.996917 8962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:16.997026 8962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:16.997200 8962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:16.997444 8962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:16.997473 8962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:16.997493 8962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:16.997504 8962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.003268 8962 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:41913
I20260806 06:46:17.003347 9095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:41913 every 8 connection(s)
I20260806 06:46:17.003662 8962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[8847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998776, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:17.012518 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 8962
I20260806 06:46:17.012626 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:17.012696 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.004396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:53237 (local address 127.6.57.126:41477)
0806 06:46:17.004486 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:17.004488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:17.004677 (+ 189us) server_negotiation.cc:407] Connection header received
0806 06:46:17.005205 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:17.005207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:17.005222 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:17.005271 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:17.006187 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.006747 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.007782 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.007993 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.008078 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.010095 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:17.010096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:17.010096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:17.010104 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:17.011328 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.011722 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.011725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.011726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.011780 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.012039 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.012041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.012041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.012153 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:17.012223 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:17.012493 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 06:46:17.012592 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:46:17.012823 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.004354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41477 (local address 127.6.57.65:53237)
0806 06:46:17.004543 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:17.004586 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:17.005078 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:17.005332 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:17.005336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:17.005872 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:17.006063 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.006068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.006842 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.006843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.007686 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.007688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.008090 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.008091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.008222 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.008670 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:17.008678 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:17.009991 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:17.011409 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.011417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.011419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.011636 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.011864 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.011865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.011865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.011970 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.012225 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:17.012233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:17.012361 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 06:46:17.012460 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:46:17.012611 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260806 06:46:17.012840 6373 external_mini_cluster.cc:1428] Setting key 9a20f3715b918085ffb100fe97eeccac
I20260806 06:46:17.013447 9096 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41477
I20260806 06:46:17.013552 9096 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:17.013746 9096 heartbeater.cc:511] Master 127.6.57.126:41477 requested a full tablet report, sending...
I20260806 06:46:17.014273 8894 ts_manager.cc:194] Registered new tserver with Master: 0e38e7f6dc4f4d4c8ebc0218ef6b2aa4 (127.6.57.65:41913)
I20260806 06:46:17.014957 8894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:53237
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:17.038369 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:17.119194 9101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:17.119377 9101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:17.119400 9101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:17.120568 9101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:17.120609 9101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:17.120635 9101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:17.122264 9101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:17.122516 9101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:17.122743 9101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:17.127724 9107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:17.127715 9109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:17.127780 9101 server_base.cc:1034] running on GCE node
W20260806 06:46:17.127715 9106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:17.128137 9101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:17.128379 9101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:17.129554 9101 hybrid_clock.cc:648] HybridClock initialized: now 1785998777129539 us; error 44 us; skew 500 ppm
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[8847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998777, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:17.132033 9101 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:17.132501 9101 webserver.cc:460] Webserver started at http://127.6.57.66:43957/ using document root <none> and password file <none>
I20260806 06:46:17.132717 9101 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:17.132759 9101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:17.132843 9101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:17.133817 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "09e0309371e74db199ad130fac855aea"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "d1d07ce8c4f8928f745a0be758db75cd"
server_key_iv: "ac0a05c08571c05262b6bd76b53a2852"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.134133 9101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "09e0309371e74db199ad130fac855aea"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "d1d07ce8c4f8928f745a0be758db75cd"
server_key_iv: "ac0a05c08571c05262b6bd76b53a2852"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.135471 9101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:17.136188 9116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.136387 9101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:17.136461 9101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "09e0309371e74db199ad130fac855aea"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "d1d07ce8c4f8928f745a0be758db75cd"
server_key_iv: "ac0a05c08571c05262b6bd76b53a2852"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.136512 9101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:17.150941 9101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:17.152732 9101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:17.152900 9101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:17.153116 9101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:17.153398 9101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:17.153431 9101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.153452 9101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:17.153496 9101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.159574 9101 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:33771
I20260806 06:46:17.159627 9236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:33771 every 8 connection(s)
I20260806 06:46:17.159938 9101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:17.163076 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9101
I20260806 06:46:17.163177 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:17.163376 6373 external_mini_cluster.cc:1428] Setting key fbfa56c2eed2b8a55e7021cd72f15fe7
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[8847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998777, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:17.169448 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.160654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:43915 (local address 127.6.57.126:41477)
0806 06:46:17.160720 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:17.160722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:17.160912 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:46:17.161416 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:17.161418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:17.161431 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:17.161484 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:17.162370 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.162921 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.164208 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.164396 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.164486 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.166562 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:17.166564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:17.166564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:17.166571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:17.168011 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.168493 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.168494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.168494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.168558 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.168838 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.168839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.168839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.168934 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:17.168984 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:17.169324 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 06:46:17.169387 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:17.169766 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.160596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41477 (local address 127.6.57.66:43915)
0806 06:46:17.160788 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:17.160829 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:17.161306 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:17.161547 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:17.161551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:17.162092 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:17.162266 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.162270 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.163019 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.163020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.164113 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.164116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.164492 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.164494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.164625 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.165127 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:17.165136 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:17.166417 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:17.168094 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.168101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.168104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.168409 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.168623 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.168624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.168624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.168766 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.169007 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:17.169013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:17.169181 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 06:46:17.169307 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 06:46:17.169482 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260806 06:46:17.170302 9237 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41477
I20260806 06:46:17.170415 9237 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:17.170607 9237 heartbeater.cc:511] Master 127.6.57.126:41477 requested a full tablet report, sending...
I20260806 06:46:17.171041 8894 ts_manager.cc:194] Registered new tserver with Master: 09e0309371e74db199ad130fac855aea (127.6.57.66:33771)
I20260806 06:46:17.171408 8894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:43915
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:17.189927 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:17.270295 9242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:17.270517 9242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:17.270541 9242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:17.271611 9242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:17.271643 9242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:17.271669 9242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:17.273002 9242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:17.273193 9242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:17.273361 9242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:17.278262 9247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:17.278285 9248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:17.278323 9242 server_base.cc:1034] running on GCE node
W20260806 06:46:17.278290 9250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:17.278738 9242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:17.278975 9242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:17.280117 9242 hybrid_clock.cc:648] HybridClock initialized: now 1785998777280061 us; error 76 us; skew 500 ppm
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[8847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998777, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:17.282624 9242 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:17.283035 9242 webserver.cc:460] Webserver started at http://127.6.57.67:38751/ using document root <none> and password file <none>
I20260806 06:46:17.283305 9242 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:17.283389 9242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:17.283541 9242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:17.284536 9242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "f483540036df476084cd7e2d789b872a"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "29d5a223cbde10ba58d705d39a3898ed"
server_key_iv: "b0687678fe65fd724552c8f6708f55a5"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.284924 9242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "f483540036df476084cd7e2d789b872a"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "29d5a223cbde10ba58d705d39a3898ed"
server_key_iv: "b0687678fe65fd724552c8f6708f55a5"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.286419 9242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:17.287333 9257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.287484 9242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:17.287578 9242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "f483540036df476084cd7e2d789b872a"
format_stamp: "Formatted at 2026-08-06 06:46:17 on dist-test-slave-5l3k"
server_key: "29d5a223cbde10ba58d705d39a3898ed"
server_key_iv: "b0687678fe65fd724552c8f6708f55a5"
server_key_version: "encryptionkey@0"
I20260806 06:46:17.287629 9242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:17.313005 9242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:17.314805 9242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:17.314917 9242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:17.315114 9242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:17.315446 9242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:17.315480 9242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.315501 9242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:17.315513 9242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:17.321231 9242 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:39729
I20260806 06:46:17.321300 9378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:39729 every 8 connection(s)
I20260806 06:46:17.321658 9242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:17.325470 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9242
I20260806 06:46:17.325544 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:17.325737 6373 external_mini_cluster.cc:1428] Setting key 03ff8809e1f43a9072fd2ff9b012b2c7
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[8847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998777, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:17.330888 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.322499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:44511 (local address 127.6.57.126:41477)
0806 06:46:17.322544 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:17.322546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:17.322765 (+ 219us) server_negotiation.cc:407] Connection header received
0806 06:46:17.323249 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:17.323250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:17.323260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:17.323317 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:17.324380 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.324889 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.325989 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.326303 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.326369 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.328430 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:17.328432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:17.328432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:17.328437 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:17.329771 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.330128 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.330129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.330129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.330179 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:17.330379 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:17.330380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:17.330380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:17.330471 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:17.330570 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:17.330772 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:46:17.330827 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:46:17.331099 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:17.322450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41477 (local address 127.6.57.67:44511)
0806 06:46:17.322637 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:17.322678 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:17.323161 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:17.323367 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:17.323370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:17.324028 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:17.324284 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.324291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.324967 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.324968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.325879 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:17.325883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:17.326389 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:17.326390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:17.326543 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:17.327089 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:17.327097 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:17.328349 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:17.329853 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.329859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.329861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.330064 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.330235 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:17.330235 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:17.330236 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:17.330326 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:17.330564 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:17.330568 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:17.330674 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 06:46:17.330758 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:46:17.330865 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260806 06:46:17.331600 9379 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41477
I20260806 06:46:17.331705 9379 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:17.331830 9379 heartbeater.cc:511] Master 127.6.57.126:41477 requested a full tablet report, sending...
I20260806 06:46:17.332170 8894 ts_manager.cc:194] Registered new tserver with Master: f483540036df476084cd7e2d789b872a (127.6.57.67:39729)
I20260806 06:46:17.332504 8894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:44511
I20260806 06:46:17.337395 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:46:17.349069 8939 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:17.348219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.6.57.126:41477)
0806 06:46:17.348279 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:17.348280 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:17.348396 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:46:17.348439 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:17.348440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:17.348449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:17.348501 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:17.348977 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42740: BlockingRecv error: recv got EOF from 127.0.0.1:42740 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:17.350024 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8962
I20260806 06:46:17.356330 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9101
I20260806 06:46:17.361366 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9242
I20260806 06:46:17.367403 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 8861
2026-08-06T06:46:17Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1858 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[9391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[9391](info): set up 2 sockets
Aug 06 06:46:17 dist-test-slave-5l3k krb5kdc[9391](info): commencing operation
krb5kdc: starting...
W20260806 06:46:18.427663 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:18.549219 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46867
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:18.629729 9406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:18.629921 9406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:18.629943 9406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:18.631026 9406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:18.631079 9406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:18.631108 9406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:18.631120 9406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:18.631132 9406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:18.632596 9406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:18.632776 9406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:18.632932 9406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:18.637658 9416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:18.637676 9414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:18.637697 9413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:18.637955 9406 server_base.cc:1034] running on GCE node
I20260806 06:46:18.638216 9406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:18.638541 9406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:18.639701 9406 hybrid_clock.cc:648] HybridClock initialized: now 1785998778639690 us; error 32 us; skew 500 ppm
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998778, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:18.642174 9406 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:18.642614 9406 webserver.cc:460] Webserver started at http://127.6.57.126:37453/ using document root <none> and password file <none>
I20260806 06:46:18.642791 9406 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:18.642832 9406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:18.642915 9406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:18.643893 9406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "78b684dca7464a1197ebbd51ebb57840"
format_stamp: "Formatted at 2026-08-06 06:46:18 on dist-test-slave-5l3k"
server_key: "31cca0cdc7c8fed33445bb223eb34711"
server_key_iv: "267ad0587c057abb80f927999262b8dc"
server_key_version: "encryptionkey@0"
I20260806 06:46:18.644196 9406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "78b684dca7464a1197ebbd51ebb57840"
format_stamp: "Formatted at 2026-08-06 06:46:18 on dist-test-slave-5l3k"
server_key: "31cca0cdc7c8fed33445bb223eb34711"
server_key_iv: "267ad0587c057abb80f927999262b8dc"
server_key_version: "encryptionkey@0"
I20260806 06:46:18.645462 9406 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:18.646137 9423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:18.646309 9406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:18.646417 9406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "78b684dca7464a1197ebbd51ebb57840"
format_stamp: "Formatted at 2026-08-06 06:46:18 on dist-test-slave-5l3k"
server_key: "31cca0cdc7c8fed33445bb223eb34711"
server_key_iv: "267ad0587c057abb80f927999262b8dc"
server_key_version: "encryptionkey@0"
I20260806 06:46:18.646509 9406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:18.675015 9406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:18.677436 9406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:18.677605 9406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:18.681445 9406 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46867
I20260806 06:46:18.681488 9478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46867 every 8 connection(s)
I20260806 06:46:18.681874 9406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:18.682786 9479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:18.683805 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9406
I20260806 06:46:18.683908 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:18.684110 6373 external_mini_cluster.cc:1428] Setting key 1be68ae7ede2d4f91e6f910814996d3b
I20260806 06:46:18.685972 9479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840: Bootstrap starting.
I20260806 06:46:18.686681 9479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:18.686951 9479 log.cc:824] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:18.687616 9479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840: No bootstrap required, opened a new log
I20260806 06:46:18.689306 9479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } }
I20260806 06:46:18.689453 9479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:18.689500 9479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b684dca7464a1197ebbd51ebb57840, State: Initialized, Role: FOLLOWER
I20260806 06:46:18.689646 9479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } }
I20260806 06:46:18.689735 9479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:18.689777 9479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:18.689848 9479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:18.690378 9479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } }
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:18.690541 9479 leader_election.cc:302] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78b684dca7464a1197ebbd51ebb57840; no voters:
I20260806 06:46:18.690721 9479 leader_election.cc:288] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:18.690765 9484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:18.690882 9484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [term 1 LEADER]: Becoming Leader. State: Replica: 78b684dca7464a1197ebbd51ebb57840, State: Running, Role: LEADER
I20260806 06:46:18.690992 9479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:18.691079 9484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } }
I20260806 06:46:18.691450 9484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78b684dca7464a1197ebbd51ebb57840. Latest consensus state: current_term: 1 leader_uuid: "78b684dca7464a1197ebbd51ebb57840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } } }
I20260806 06:46:18.691596 9484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:18.691826 9494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:18.691823 9487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78b684dca7464a1197ebbd51ebb57840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b684dca7464a1197ebbd51ebb57840" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46867 } } }
I20260806 06:46:18.692413 9494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:18.693128 9487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:18.694046 9494 catalog_manager.cc:1347] Generated new cluster ID: efab843b60a34b8ca26cfecd1b2b94be
I20260806 06:46:18.694157 9494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:18.694669 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:18.684649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46088 (local address 127.6.57.126:46867)
0806 06:46:18.684857 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:18.684865 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:18.684890 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:46:18.685343 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:18.685350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:18.685395 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:18.685539 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:18.686190 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:18.687237 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:18.688028 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:18.688424 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:18.688543 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:18.691009 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:18.691012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:18.691013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:18.691024 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:18.692921 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:18.693345 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:18.693348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:18.693348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:18.693480 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:18.693707 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:18.693708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:18.693708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:18.693996 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:18.694157 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:18.694309 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:46:18.694446 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260806 06:46:18.695572 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:46:18.699404 9494 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:18.699846 9494 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:18.704770 9494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78b684dca7464a1197ebbd51ebb57840: Generated new TSK 0
I20260806 06:46:18.704921 9494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:18.711853 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:18.708356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.6.57.126:46867)
0806 06:46:18.708411 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:18.708413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:18.708496 (+ 83us) server_negotiation.cc:407] Connection header received
0806 06:46:18.708559 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:18.708560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:18.708570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:18.708629 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:18.708927 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:18.709671 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:18.710354 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:18.710704 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:18.710771 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:18.711389 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:18.711391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:18.711391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:18.711620 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:18.711636 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:18.711642 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:18.711699 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:18.711760 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:46:18.711784 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:46:18.714154 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9406
2026-08-06T06:46:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1345 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9514](info): set up 2 sockets
Aug 06 06:46:18 dist-test-slave-5l3k krb5kdc[9514](info): commencing operation
krb5kdc: starting...
W20260806 06:46:19.764309 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:19 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:19.885641 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46777
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:19.968535 9527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:19.968766 9527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:19.968828 9527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:19.970007 9527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:19.970091 9527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:19.970124 9527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:19.970157 9527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:19.970198 9527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:19.971720 9527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:19.972023 9527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:19.972270 9527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:19.977133 9532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:19.977128 9535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:19.977195 9533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:19.977494 9527 server_base.cc:1034] running on GCE node
I20260806 06:46:19.977766 9527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:19.978082 9527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:19.979267 9527 hybrid_clock.cc:648] HybridClock initialized: now 1785998779979246 us; error 47 us; skew 500 ppm
Aug 06 06:46:19 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998779, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:19.981752 9527 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:19.982189 9527 webserver.cc:460] Webserver started at http://127.6.57.126:35959/ using document root <none> and password file <none>
I20260806 06:46:19.982445 9527 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:19.982553 9527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:19.982700 9527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:19.983743 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67"
format_stamp: "Formatted at 2026-08-06 06:46:19 on dist-test-slave-5l3k"
server_key: "720071ce22d163248b92d1b59e25c72c"
server_key_iv: "a846da973ead5eeb7be1dc1119364844"
server_key_version: "encryptionkey@0"
I20260806 06:46:19.984084 9527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67"
format_stamp: "Formatted at 2026-08-06 06:46:19 on dist-test-slave-5l3k"
server_key: "720071ce22d163248b92d1b59e25c72c"
server_key_iv: "a846da973ead5eeb7be1dc1119364844"
server_key_version: "encryptionkey@0"
I20260806 06:46:19.985363 9527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:19.986088 9542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:19.986258 9527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:19.986356 9527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67"
format_stamp: "Formatted at 2026-08-06 06:46:19 on dist-test-slave-5l3k"
server_key: "720071ce22d163248b92d1b59e25c72c"
server_key_iv: "a846da973ead5eeb7be1dc1119364844"
server_key_version: "encryptionkey@0"
I20260806 06:46:19.986430 9527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:19.997555 9527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:19.999434 9527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:19.999636 9527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:20.003621 9527 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46777
I20260806 06:46:20.003685 9596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46777 every 8 connection(s)
I20260806 06:46:20.004029 9527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:20.005074 9597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:20.007467 9597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67: Bootstrap starting.
I20260806 06:46:20.008064 9597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:20.008356 9597 log.cc:824] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:20.009068 9597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67: No bootstrap required, opened a new log
I20260806 06:46:20.010393 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9527
I20260806 06:46:20.010522 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:20.010711 6373 external_mini_cluster.cc:1428] Setting key 582a5be408fb490ea1b8fb9fb40fed06
I20260806 06:46:20.010811 9597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } }
I20260806 06:46:20.010974 9597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:20.011024 9597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50828e2d0bc442fe8d0f3ff7cee5ec67, State: Initialized, Role: FOLLOWER
I20260806 06:46:20.011189 9597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } }
I20260806 06:46:20.011296 9597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:20.011355 9597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:20.011415 9597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:20.012640 9597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } }
I20260806 06:46:20.012779 9597 leader_election.cc:302] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50828e2d0bc442fe8d0f3ff7cee5ec67; no voters:
I20260806 06:46:20.012951 9597 leader_election.cc:288] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:20.012985 9602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:20.013087 9602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [term 1 LEADER]: Becoming Leader. State: Replica: 50828e2d0bc442fe8d0f3ff7cee5ec67, State: Running, Role: LEADER
I20260806 06:46:20.013176 9602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } }
I20260806 06:46:20.013206 9597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:20.013480 9602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } } }
I20260806 06:46:20.013548 9602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:20.013705 9602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50828e2d0bc442fe8d0f3ff7cee5ec67. Latest consensus state: current_term: 1 leader_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50828e2d0bc442fe8d0f3ff7cee5ec67" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46777 } } }
I20260806 06:46:20.013840 9602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:20.014262 9606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:20.014853 9606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:20.016633 9606 catalog_manager.cc:1347] Generated new cluster ID: 3087c9e9a7e14b47aa2c91c269859b2c
I20260806 06:46:20.016695 9606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:20.023460 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.011285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.6.57.126:46777)
0806 06:46:20.011494 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:20.011503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:20.011557 (+ 54us) server_negotiation.cc:407] Connection header received
0806 06:46:20.011657 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:20.011662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:20.011703 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:20.011814 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:20.012692 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.014140 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.015341 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.015904 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.016036 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.018897 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:20.018902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:20.018902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:20.018915 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:20.021034 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.021486 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.021489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.021489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.022268 (+ 779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.022518 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.022519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.022519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.022836 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:20.023011 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:20.023121 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:46:20.023264 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
I20260806 06:46:20.029457 9606 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:20.030176 9606 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:20.037465 9606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50828e2d0bc442fe8d0f3ff7cee5ec67: Generated new TSK 0
I20260806 06:46:20.037643 9606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:20.068938 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:20.150796 9625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:20.150983 9625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:20.151006 9625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:20.152137 9625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:20.152179 9625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:20.152230 9625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:20.153728 9625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:20.153976 9625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:20.154215 9625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:20.159173 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.159186 9631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.159178 9630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:20.159423 9625 server_base.cc:1034] running on GCE node
I20260806 06:46:20.159600 9625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:20.159790 9625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:20.160925 9625 hybrid_clock.cc:648] HybridClock initialized: now 1785998780160910 us; error 31 us; skew 500 ppm
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:20.163365 9625 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:20.163790 9625 webserver.cc:460] Webserver started at http://127.6.57.65:42655/ using document root <none> and password file <none>
I20260806 06:46:20.164032 9625 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:20.164103 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:20.164261 9625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:20.165239 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "884b304ae2ed467290cafd08f0f11b5d"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "30b5d20652f53090d6e807a7fad0dfea"
server_key_iv: "8cf28c84b7c553dd6df49740e6d4be6b"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.165578 9625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "884b304ae2ed467290cafd08f0f11b5d"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "30b5d20652f53090d6e807a7fad0dfea"
server_key_iv: "8cf28c84b7c553dd6df49740e6d4be6b"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.166899 9625 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:20.167706 9640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.167865 9625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:20.167968 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "884b304ae2ed467290cafd08f0f11b5d"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "30b5d20652f53090d6e807a7fad0dfea"
server_key_iv: "8cf28c84b7c553dd6df49740e6d4be6b"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.168037 9625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:20.181838 9625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:20.183975 9625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:20.184149 9625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:20.184379 9625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:20.184695 9625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:20.184762 9625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.184792 9625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:20.184820 9625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.190763 9625 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39511
I20260806 06:46:20.190836 9760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39511 every 8 connection(s)
I20260806 06:46:20.191141 9625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:20.193845 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9625
I20260806 06:46:20.193940 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:20.194116 6373 external_mini_cluster.cc:1428] Setting key 1a9ff82c78df1abafcc22d8dd0faf5c0
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:20.201702 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.191790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:56397 (local address 127.6.57.126:46777)
0806 06:46:20.191854 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:20.191856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:20.192074 (+ 218us) server_negotiation.cc:407] Connection header received
0806 06:46:20.192579 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:20.192581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:20.192595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:20.192652 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:20.193551 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.194385 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.195549 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.195879 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.195956 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.198689 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:20.198691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:20.198691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:20.198700 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:20.200151 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.200634 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.200636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.200636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.200709 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.200972 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.200974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.200974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.201104 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:20.201177 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:20.201499 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 06:46:20.201605 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:20.201910 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.191756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46777 (local address 127.6.57.65:56397)
0806 06:46:20.191948 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:20.191990 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:20.192494 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:20.192705 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:20.192708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:20.193243 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:20.193444 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.193449 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.194521 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.194522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.195445 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.195448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.195966 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.195966 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.196069 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.196596 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:20.196607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:20.198589 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:20.200245 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.200251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.200253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.200526 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.200782 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.200783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.200783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.200892 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.201215 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:20.201224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:20.201350 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 06:46:20.201476 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 06:46:20.201664 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260806 06:46:20.202451 9761 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46777
I20260806 06:46:20.202592 9761 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:20.202803 9761 heartbeater.cc:511] Master 127.6.57.126:46777 requested a full tablet report, sending...
I20260806 06:46:20.203425 9560 ts_manager.cc:194] Registered new tserver with Master: 884b304ae2ed467290cafd08f0f11b5d (127.6.57.65:39511)
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:46:20.204385 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:56397
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:20.221554 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:20.305074 9767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:20.305316 9767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:20.305346 9767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:20.307087 9767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:20.307147 9767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:20.307191 9767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:20.308736 9767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:20.308990 9767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:20.309260 9767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:20.313983 9772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.313995 9773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.314179 9775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:20.314221 9767 server_base.cc:1034] running on GCE node
I20260806 06:46:20.314419 9767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:20.314674 9767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:20.315876 9767 hybrid_clock.cc:648] HybridClock initialized: now 1785998780315868 us; error 47 us; skew 500 ppm
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:20.318694 9767 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:20.319222 9767 webserver.cc:460] Webserver started at http://127.6.57.66:33795/ using document root <none> and password file <none>
I20260806 06:46:20.319480 9767 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:20.319558 9767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:20.319718 9767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:20.320887 9767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "6737ff9b47844e519967423f9413bb25"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "bedad0468ecafb9889c1b783acde3535"
server_key_iv: "e8b4c82c8efd099c4f0c632e8fe5c74c"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.321319 9767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "6737ff9b47844e519967423f9413bb25"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "bedad0468ecafb9889c1b783acde3535"
server_key_iv: "e8b4c82c8efd099c4f0c632e8fe5c74c"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.322868 9767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:20.323647 9782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.323832 9767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:20.323941 9767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "6737ff9b47844e519967423f9413bb25"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "bedad0468ecafb9889c1b783acde3535"
server_key_iv: "e8b4c82c8efd099c4f0c632e8fe5c74c"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.324018 9767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:20.335043 9767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:20.336905 9767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:20.337060 9767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:20.337278 9767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:20.337580 9767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:20.337633 9767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.337663 9767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:20.337690 9767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.343405 9767 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:39611
I20260806 06:46:20.343503 9902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:39611 every 8 connection(s)
I20260806 06:46:20.343786 9767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:20.347044 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9767
I20260806 06:46:20.347119 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:20.347285 6373 external_mini_cluster.cc:1428] Setting key 94f0fa6ca4e0d1b2a3eb9da986f41f1f
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:20.352808 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.344454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:54701 (local address 127.6.57.126:46777)
0806 06:46:20.344499 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:20.344501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:20.344708 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:46:20.345134 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:20.345135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:20.345144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:20.345197 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:20.346030 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.346532 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.347554 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.347798 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.347874 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.349839 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:20.349840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:20.349840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:20.349847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:20.351211 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.351718 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.351718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.351719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.351783 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.352088 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.352089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.352089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.352264 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:20.352335 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:20.352683 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 06:46:20.352746 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:20.353093 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.344420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46777 (local address 127.6.57.66:54701)
0806 06:46:20.344584 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:20.344623 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:20.345055 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:20.345244 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:20.345247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:20.345789 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:20.345958 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.345962 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.346600 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.346601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.347433 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.347435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.347891 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.347892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.348012 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.348498 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:20.348506 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:20.349746 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:20.351311 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.351317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.351319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.351644 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.351866 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.351867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.351868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.352018 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.352352 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:20.352359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:20.352526 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:46:20.352654 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:46:20.352824 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260806 06:46:20.353658 9903 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46777
I20260806 06:46:20.353749 9903 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:20.353929 9903 heartbeater.cc:511] Master 127.6.57.126:46777 requested a full tablet report, sending...
I20260806 06:46:20.354291 9560 ts_manager.cc:194] Registered new tserver with Master: 6737ff9b47844e519967423f9413bb25 (127.6.57.66:39611)
I20260806 06:46:20.354664 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:54701
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:20.373276 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:20.452411 9908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:20.452620 9908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:20.452656 9908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:20.453758 9908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:20.453819 9908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:20.453866 9908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:20.455384 9908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:20.455632 9908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:20.455845 9908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:46:20.460778 9908 server_base.cc:1034] running on GCE node
W20260806 06:46:20.460731 9914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.460762 9917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:20.460731 9913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:20.461086 9908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:20.461328 9908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:20.462527 9908 hybrid_clock.cc:648] HybridClock initialized: now 1785998780462517 us; error 31 us; skew 500 ppm
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:20.464969 9908 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:20.465415 9908 webserver.cc:460] Webserver started at http://127.6.57.67:43629/ using document root <none> and password file <none>
I20260806 06:46:20.465711 9908 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:20.465759 9908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:20.465910 9908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:20.466969 9908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "cb487ae46130462aaee4af96e66927d9"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "26bc39eb77c1b059d85f349bccf9dcbe"
server_key_iv: "187d2d5c01fe4d834a5b8cc215b15550"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.467319 9908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "cb487ae46130462aaee4af96e66927d9"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "26bc39eb77c1b059d85f349bccf9dcbe"
server_key_iv: "187d2d5c01fe4d834a5b8cc215b15550"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.468591 9908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:20.469308 9924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.469503 9908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:20.469587 9908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "cb487ae46130462aaee4af96e66927d9"
format_stamp: "Formatted at 2026-08-06 06:46:20 on dist-test-slave-5l3k"
server_key: "26bc39eb77c1b059d85f349bccf9dcbe"
server_key_iv: "187d2d5c01fe4d834a5b8cc215b15550"
server_key_version: "encryptionkey@0"
I20260806 06:46:20.469638 9908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:20.477036 9908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:20.478854 9908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:20.478957 9908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:20.479123 9908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:20.479424 9908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:20.479457 9908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.479506 9908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:20.479533 9908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:20.485163 9908 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:36577
I20260806 06:46:20.485255 10042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:36577 every 8 connection(s)
I20260806 06:46:20.485548 9908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:20.488050 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 9908
I20260806 06:46:20.488155 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:20.488330 6373 external_mini_cluster.cc:1428] Setting key 0c9613c15deb9a73f2751eb1e6d3f694
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:20.494902 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.486272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:39605 (local address 127.6.57.126:46777)
0806 06:46:20.486330 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:20.486332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:20.486538 (+ 206us) server_negotiation.cc:407] Connection header received
0806 06:46:20.487015 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:20.487016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:20.487027 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:20.487092 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:20.487972 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.488722 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.489849 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.490025 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.490085 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.492137 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:20.492138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:20.492138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:20.492146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:20.493458 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.493879 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.493881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.493881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.493947 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.494207 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.494208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.494208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.494326 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:20.494401 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:20.494729 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 06:46:20.494811 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:20.495092 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.486228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46777 (local address 127.6.57.67:39605)
0806 06:46:20.486392 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:20.486431 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:20.486917 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:20.487144 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:20.487147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:20.487679 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:20.487888 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.487893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.488819 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.488820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.489782 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:20.489785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.490102 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.490103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:20.490214 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.490772 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:20.490780 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:20.492004 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:20.493573 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.493579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.493582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.493798 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.494017 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:20.494018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:20.494018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:20.494129 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:20.494392 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:20.494399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:20.494596 (+ 197us) client_negotiation.cc:769] Sending connection context
0806 06:46:20.494707 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 06:46:20.494859 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260806 06:46:20.495589 10043 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46777
I20260806 06:46:20.495682 10043 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:20.495852 10043 heartbeater.cc:511] Master 127.6.57.126:46777 requested a full tablet report, sending...
I20260806 06:46:20.496234 9560 ts_manager.cc:194] Registered new tserver with Master: cb487ae46130462aaee4af96e66927d9 (127.6.57.67:36577)
I20260806 06:46:20.496598 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:39605
I20260806 06:46:20.500063 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[9514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998780, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:20.523026 9601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:20.516195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.6.57.126:46777)
0806 06:46:20.516263 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:20.516265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:20.516413 (+ 148us) server_negotiation.cc:407] Connection header received
0806 06:46:20.516456 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:20.516457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:20.516467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:20.516514 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:20.517089 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.517597 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.518326 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:20.518583 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:20.518642 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:20.520829 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:20.520830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:20.520830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:20.520837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:20.521993 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.522310 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.522310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.522311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.522361 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:20.522570 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:20.522571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:20.522571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:20.522705 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:20.522786 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:20.522869 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:46:20.522925 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:46:20.523321 9560 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34112
I20260806 06:46:20.523793 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9625
I20260806 06:46:20.529947 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9767
I20260806 06:46:20.535234 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9908
I20260806 06:46:20.540699 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 9527
2026-08-06T06:46:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1828 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[10057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[10057](info): set up 2 sockets
Aug 06 06:46:20 dist-test-slave-5l3k krb5kdc[10057](info): commencing operation
krb5kdc: starting...
W20260806 06:46:21.645076 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:21 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:21.761765 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:38439
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:38439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:21.842614 10070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:21.842839 10070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:21.842877 10070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:21.844022 10070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:21.844094 10070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:21.844120 10070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:21.844152 10070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:21.844177 10070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:21.845695 10070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:38439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:38439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:21.845969 10070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:21.846202 10070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:46:21.850975 10070 server_base.cc:1034] running on GCE node
W20260806 06:46:21.850939 10079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:21.850947 10076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:21.850956 10075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:21.851382 10070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:21.851665 10070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:21.852836 10070 hybrid_clock.cc:648] HybridClock initialized: now 1785998781852826 us; error 41 us; skew 500 ppm
Aug 06 06:46:21 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998781, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:21.855472 10070 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:21.855979 10070 webserver.cc:460] Webserver started at http://127.6.57.126:38397/ using document root <none> and password file <none>
I20260806 06:46:21.856236 10070 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:21.856307 10070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:21.856459 10070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:21.857517 10070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "032a43e59a5648aa9f166ecbdf24a68e"
format_stamp: "Formatted at 2026-08-06 06:46:21 on dist-test-slave-5l3k"
server_key: "8ab226f0a67cb99dbf84ea69a3f01e13"
server_key_iv: "b5210bb8728cf678317103046bf5e126"
server_key_version: "encryptionkey@0"
I20260806 06:46:21.857913 10070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "032a43e59a5648aa9f166ecbdf24a68e"
format_stamp: "Formatted at 2026-08-06 06:46:21 on dist-test-slave-5l3k"
server_key: "8ab226f0a67cb99dbf84ea69a3f01e13"
server_key_iv: "b5210bb8728cf678317103046bf5e126"
server_key_version: "encryptionkey@0"
I20260806 06:46:21.859326 10070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260806 06:46:21.860037 10086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:21.860203 10070 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:21.860306 10070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "032a43e59a5648aa9f166ecbdf24a68e"
format_stamp: "Formatted at 2026-08-06 06:46:21 on dist-test-slave-5l3k"
server_key: "8ab226f0a67cb99dbf84ea69a3f01e13"
server_key_iv: "b5210bb8728cf678317103046bf5e126"
server_key_version: "encryptionkey@0"
I20260806 06:46:21.860378 10070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:21.872334 10070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:21.874168 10070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:21.874326 10070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:21.878264 10070 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:38439
I20260806 06:46:21.878314 10141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:38439 every 8 connection(s)
I20260806 06:46:21.878873 10070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:21.879523 10142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:21.881742 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e: Bootstrap starting.
I20260806 06:46:21.882366 10142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:21.882686 10142 log.cc:824] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:21.883347 10142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e: No bootstrap required, opened a new log
I20260806 06:46:21.884920 10142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } }
I20260806 06:46:21.885061 10142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:21.885082 10142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032a43e59a5648aa9f166ecbdf24a68e, State: Initialized, Role: FOLLOWER
I20260806 06:46:21.885248 10142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } }
I20260806 06:46:21.885308 10142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:21.885329 10142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:21.885396 10142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:21.886225 10142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } }
I20260806 06:46:21.886337 10142 leader_election.cc:302] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 032a43e59a5648aa9f166ecbdf24a68e; no voters:
I20260806 06:46:21.886562 10142 leader_election.cc:288] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:21.886602 10145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:21.886694 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10070
I20260806 06:46:21.886788 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:21.886782 10145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [term 1 LEADER]: Becoming Leader. State: Replica: 032a43e59a5648aa9f166ecbdf24a68e, State: Running, Role: LEADER
I20260806 06:46:21.886874 10142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:21.886891 10145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } }
I20260806 06:46:21.886958 6373 external_mini_cluster.cc:1428] Setting key a0980cda8c5693b795aec04389da3439
I20260806 06:46:21.887229 10145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 032a43e59a5648aa9f166ecbdf24a68e. Latest consensus state: current_term: 1 leader_uuid: "032a43e59a5648aa9f166ecbdf24a68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } } }
I20260806 06:46:21.887236 10146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "032a43e59a5648aa9f166ecbdf24a68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "032a43e59a5648aa9f166ecbdf24a68e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 38439 } } }
I20260806 06:46:21.887317 10145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:21.887321 10146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:21.887863 10152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:21.888433 10152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:21.890096 10152 catalog_manager.cc:1347] Generated new cluster ID: 3d77b2ee1af64ee0bc10633667b12426
I20260806 06:46:21.890147 10152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:21 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:21.898809 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:21.887712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.6.57.126:38439)
0806 06:46:21.888024 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:21.888033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:21.888069 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:46:21.888218 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:21.888225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:21.888290 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:21.888421 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:21.889211 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:21.890526 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:21.891566 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:21.892175 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:21.892267 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:21.895087 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:21.895091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:21.895092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:21.895104 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:21.897123 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:21.897514 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:21.897518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:21.897518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:21.897636 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:21.897851 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:21.897852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:21.897853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:21.898189 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:21.898356 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:21.898424 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:46:21.898583 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":78,"threads_started":1}
I20260806 06:46:21.911856 10152 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:21.912567 10152 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:21.915004 10152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 032a43e59a5648aa9f166ecbdf24a68e: Generated new TSK 0
I20260806 06:46:21.915161 10152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:21.944751 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:22.025368 10169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:22.025557 10169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:22.025579 10169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:22.026742 10169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:22.026783 10169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:22.026808 10169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:22.028259 10169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:22.028497 10169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:22.028760 10169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:22.033773 10174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.033782 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.033798 10177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:22.034018 10169 server_base.cc:1034] running on GCE node
I20260806 06:46:22.034207 10169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:22.034444 10169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:22.035627 10169 hybrid_clock.cc:648] HybridClock initialized: now 1785998782035614 us; error 40 us; skew 500 ppm
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:22.038115 10169 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:22.038589 10169 webserver.cc:460] Webserver started at http://127.6.57.65:40741/ using document root <none> and password file <none>
I20260806 06:46:22.038834 10169 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:22.038905 10169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:22.039055 10169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:22.040068 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "45d3ff7e70804b858fb05d6cc46d02b3"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "e6086abce01734bdc39ade9249b7c0ef"
server_key_iv: "709ad74f67efe03d42e7ae37bd3b32ec"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.040412 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "45d3ff7e70804b858fb05d6cc46d02b3"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "e6086abce01734bdc39ade9249b7c0ef"
server_key_iv: "709ad74f67efe03d42e7ae37bd3b32ec"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.041747 10169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:22.042567 10184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.042775 10169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:22.042870 10169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "45d3ff7e70804b858fb05d6cc46d02b3"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "e6086abce01734bdc39ade9249b7c0ef"
server_key_iv: "709ad74f67efe03d42e7ae37bd3b32ec"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.042955 10169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:22.064905 10169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:22.066740 10169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:22.066895 10169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:22.067138 10169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:22.067448 10169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:22.067502 10169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.067540 10169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:22.067567 10169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.073215 10169 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:42345
I20260806 06:46:22.073266 10300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:42345 every 8 connection(s)
I20260806 06:46:22.073552 10169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:22.079778 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10169
I20260806 06:46:22.079887 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:22.080102 6373 external_mini_cluster.cc:1428] Setting key cc224096ca3d1e97e9b0f4b8639deac5
I20260806 06:46:22.082957 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.074249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:52021 (local address 127.6.57.126:38439)
0806 06:46:22.074320 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:22.074322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:22.074518 (+ 196us) server_negotiation.cc:407] Connection header received
0806 06:46:22.075440 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:22.075441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:22.075455 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:22.075515 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:22.076036 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.076554 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.077518 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.077710 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.077790 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.079766 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:22.079768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:22.079769 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:22.079778 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:22.081538 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.081906 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.081907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.081908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.081970 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.082192 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.082193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.082193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.082356 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:22.082427 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:22.082776 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 06:46:22.082867 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:46:22.083134 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.074225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:38439 (local address 127.6.57.65:52021)
0806 06:46:22.074380 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:22.074419 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:22.075353 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:22.075565 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:22.075567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:22.075793 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:22.075965 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.075968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.076646 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.076647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.077433 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.077435 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.077811 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.077812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.077905 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.078301 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:22.078309 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:22.079666 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:22.081623 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.081629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.081630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.081832 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.082036 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.082036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.082036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.082130 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.082421 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:22.082425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:22.082588 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 06:46:22.082735 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 06:46:22.082890 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260806 06:46:22.083653 10301 heartbeater.cc:347] Connected to a master server at 127.6.57.126:38439
I20260806 06:46:22.083755 10301 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:22.083976 10301 heartbeater.cc:511] Master 127.6.57.126:38439 requested a full tablet report, sending...
I20260806 06:46:22.084462 10104 ts_manager.cc:194] Registered new tserver with Master: 45d3ff7e70804b858fb05d6cc46d02b3 (127.6.57.65:42345)
I20260806 06:46:22.085067 10104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:52021
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:22.105486 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:22.184619 10307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:22.184808 10307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:22.184831 10307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:22.185891 10307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:22.185926 10307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:22.185951 10307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:22.187513 10307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:22.187721 10307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:22.187919 10307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:22.192751 10315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.192778 10312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.192751 10313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:22.192919 10307 server_base.cc:1034] running on GCE node
I20260806 06:46:22.193163 10307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:22.193400 10307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:22.194542 10307 hybrid_clock.cc:648] HybridClock initialized: now 1785998782194527 us; error 24 us; skew 500 ppm
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:22.197420 10307 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:22.197849 10307 webserver.cc:460] Webserver started at http://127.6.57.66:40383/ using document root <none> and password file <none>
I20260806 06:46:22.198066 10307 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:22.198109 10307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:22.198252 10307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:22.199338 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "43a8db8b06de4c73b31b1d1a112bcf1b"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "1b57be1da8348f0e1acc6d6ec3493fc7"
server_key_iv: "20565028405347036483891bba4c9a60"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.199688 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "43a8db8b06de4c73b31b1d1a112bcf1b"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "1b57be1da8348f0e1acc6d6ec3493fc7"
server_key_iv: "20565028405347036483891bba4c9a60"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.200974 10307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:22.201671 10322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.201844 10307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:22.201927 10307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "43a8db8b06de4c73b31b1d1a112bcf1b"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "1b57be1da8348f0e1acc6d6ec3493fc7"
server_key_iv: "20565028405347036483891bba4c9a60"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.201995 10307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:22.212633 10307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:22.214867 10307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:22.215035 10307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:22.215250 10307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:22.215536 10307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:22.215588 10307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.215633 10307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:22.215660 10307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.221551 10307 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:41655
I20260806 06:46:22.221601 10438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:41655 every 8 connection(s)
I20260806 06:46:22.221962 10307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:22.230437 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10307
I20260806 06:46:22.230576 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:22.230822 6373 external_mini_cluster.cc:1428] Setting key 317d9437821ea52430e64744e96315ed
I20260806 06:46:22.231753 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.222825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:55617 (local address 127.6.57.126:38439)
0806 06:46:22.222885 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:22.222887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:22.223153 (+ 266us) server_negotiation.cc:407] Connection header received
0806 06:46:22.224081 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:22.224082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:22.224092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:22.224154 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:22.224710 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.225216 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.226261 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.226447 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.226568 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.228699 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:22.228701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:22.228701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:22.228710 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:22.230019 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.230575 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.230577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.230577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.230654 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.230929 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.230930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.230931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.231062 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:22.231153 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:22.231510 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 06:46:22.231644 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:46:22.231848 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.222789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:38439 (local address 127.6.57.66:55617)
0806 06:46:22.222964 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:22.223007 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:22.223988 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:22.224206 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:22.224210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:22.224449 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:22.224640 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.224644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.225358 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.225359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.226180 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.226183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.226611 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.226613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.226716 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.227159 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:22.227168 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:22.228541 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:22.230107 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.230114 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.230117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.230461 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.230750 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.230751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.230751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.230854 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.231190 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:22.231198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:22.231358 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 06:46:22.231474 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:46:22.231649 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260806 06:46:22.232524 10439 heartbeater.cc:347] Connected to a master server at 127.6.57.126:38439
I20260806 06:46:22.232617 10439 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:22.232789 10439 heartbeater.cc:511] Master 127.6.57.126:38439 requested a full tablet report, sending...
I20260806 06:46:22.233156 10104 ts_manager.cc:194] Registered new tserver with Master: 43a8db8b06de4c73b31b1d1a112bcf1b (127.6.57.66:41655)
I20260806 06:46:22.233693 10104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:55617
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:22.256963 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:22.337970 10445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:22.338191 10445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:22.338230 10445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:22.339417 10445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:22.339481 10445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:22.339525 10445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:22.340968 10445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:38439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:22.341239 10445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:22.341466 10445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:22.346303 10450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.346354 10451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:22.346303 10453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:22.346558 10445 server_base.cc:1034] running on GCE node
I20260806 06:46:22.346777 10445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:22.347005 10445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:22.348178 10445 hybrid_clock.cc:648] HybridClock initialized: now 1785998782348170 us; error 30 us; skew 500 ppm
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:22.350673 10445 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:22.351147 10445 webserver.cc:460] Webserver started at http://127.6.57.67:34337/ using document root <none> and password file <none>
I20260806 06:46:22.351382 10445 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:22.351444 10445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:22.351589 10445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:22.352545 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "ce0d751b521c4b86803f1bc2f8015024"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "5984adb56a66c1ab82178e6783f58c87"
server_key_iv: "9745c726bc70eb4be757111ff2a1af49"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.352870 10445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "ce0d751b521c4b86803f1bc2f8015024"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "5984adb56a66c1ab82178e6783f58c87"
server_key_iv: "9745c726bc70eb4be757111ff2a1af49"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.354094 10445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:22.354876 10460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.355069 10445 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.355177 10445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "ce0d751b521c4b86803f1bc2f8015024"
format_stamp: "Formatted at 2026-08-06 06:46:22 on dist-test-slave-5l3k"
server_key: "5984adb56a66c1ab82178e6783f58c87"
server_key_iv: "9745c726bc70eb4be757111ff2a1af49"
server_key_version: "encryptionkey@0"
I20260806 06:46:22.355264 10445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:22.379637 10445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:22.381451 10445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:22.381603 10445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:22.381816 10445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:22.382133 10445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:22.382191 10445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.382229 10445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:22.382254 10445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:22.388290 10445 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:33845
I20260806 06:46:22.388371 10578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:33845 every 8 connection(s)
I20260806 06:46:22.388674 10445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:22.391829 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10445
I20260806 06:46:22.391925 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:22.392122 6373 external_mini_cluster.cc:1428] Setting key 73ae879f404ceb81a83da44da9dfa6ad
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:22.397899 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.389406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:55555 (local address 127.6.57.126:38439)
0806 06:46:22.389447 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:22.389449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:22.389663 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:46:22.390645 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:22.390646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:22.390660 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:22.390720 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:22.391321 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.391954 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.393228 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.393442 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.393528 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.395531 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:22.395532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:22.395533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:22.395540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:22.396603 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.396970 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.396971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.396971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.397037 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.397280 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.397281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.397281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.397385 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:22.397455 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:22.397704 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 06:46:22.397813 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:46:22.398041 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.389387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:38439 (local address 127.6.57.67:55555)
0806 06:46:22.389536 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:22.389589 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:22.390528 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:22.390783 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:22.390787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:22.391045 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:22.391216 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.391220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.392054 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.392055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.393140 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:22.393143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.393540 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.393541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:22.393662 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.394065 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:22.394073 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:22.395422 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:22.396690 (+ 1268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.396697 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.396699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.396896 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.397104 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:22.397105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:22.397105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:22.397209 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:22.397446 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:22.397452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:22.397575 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:46:22.397669 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:46:22.397836 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260806 06:46:22.398530 10579 heartbeater.cc:347] Connected to a master server at 127.6.57.126:38439
I20260806 06:46:22.398640 10579 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:22.398856 10579 heartbeater.cc:511] Master 127.6.57.126:38439 requested a full tablet report, sending...
I20260806 06:46:22.399246 10104 ts_manager.cc:194] Registered new tserver with Master: ce0d751b521c4b86803f1bc2f8015024 (127.6.57.67:33845)
I20260806 06:46:22.399621 10104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:55555
I20260806 06:46:22.404175 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:22.427668 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.420316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40372 (local address 127.6.57.65:42345)
0806 06:46:22.420480 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:22.420484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:22.420492 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:22.420530 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:22.420532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:22.420545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:22.420633 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:22.421192 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.421870 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.422646 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.422933 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.423011 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.425099 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:22.425108 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:22.425118 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:22.425125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:22.426462 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.426866 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.426869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.426869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.426932 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.427145 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.427146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.427147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.427277 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:22.427430 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:22.427479 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:46:22.427556 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:22.434955 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:22.428768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.6.57.126:38439)
0806 06:46:22.428821 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:22.428822 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:22.428901 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:46:22.428935 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:22.428936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:22.428943 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:22.428979 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:22.429360 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.429768 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.430349 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:22.430559 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:22.430608 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:22.432531 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:22.432532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:22.432532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:22.432538 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:22.433676 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.434056 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.434057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.434058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.434118 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:22.434324 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:22.434325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:22.434325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:22.434469 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:22.434618 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:22.434794 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:46:22.434888 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 06:46:22.435205 10104 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60978
I20260806 06:46:22.435529 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10169
I20260806 06:46:22.443393 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10307
I20260806 06:46:22.448449 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10445
I20260806 06:46:22.456947 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10070
2026-08-06T06:46:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1916 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10591](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10591](info): set up 2 sockets
Aug 06 06:46:22 dist-test-slave-5l3k krb5kdc[10591](info): commencing operation
krb5kdc: starting...
W20260806 06:46:23.504375 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:23 dist-test-slave-5l3k krb5kdc[10591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:23.621642 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41589
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:23.707343 10604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:23.707540 10604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:23.707562 10604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:23.708604 10604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:23.708638 10604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:23.708652 10604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:23.708663 10604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:23.708674 10604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:23.710028 10604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:23.710232 10604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:23.710386 10604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:23.714969 10612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:23.714949 10610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:23.715266 10604 server_base.cc:1034] running on GCE node
W20260806 06:46:23.714956 10609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:23.715583 10604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:23.715871 10604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:23.717034 10604 hybrid_clock.cc:648] HybridClock initialized: now 1785998783717020 us; error 43 us; skew 500 ppm
Aug 06 06:46:23 dist-test-slave-5l3k krb5kdc[10591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998783, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:23.719519 10604 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:23.719803 10604 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:23.719843 10604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:23.719952 10604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:23.720898 10604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "75d4a38bd01a4944b616b512e9a5e17a"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "acd8fdadf8caebd19f377aceed8c3de3"
server_key_iv: "f2af94121fcade4f17ae28a521badc25"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.721194 10604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "75d4a38bd01a4944b616b512e9a5e17a"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "acd8fdadf8caebd19f377aceed8c3de3"
server_key_iv: "f2af94121fcade4f17ae28a521badc25"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.722445 10604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:23.723237 10620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:23.723395 10604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:23.723481 10604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "75d4a38bd01a4944b616b512e9a5e17a"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "acd8fdadf8caebd19f377aceed8c3de3"
server_key_iv: "f2af94121fcade4f17ae28a521badc25"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.723537 10604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:23.733727 10604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:23.735807 10604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:23.735935 10604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:23.739864 10604 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41589
I20260806 06:46:23.739893 10673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41589 every 8 connection(s)
I20260806 06:46:23.740200 10604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:23.741240 10674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:23.743661 10674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: Bootstrap starting.
I20260806 06:46:23.744334 10674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:23.744654 10674 log.cc:824] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:23.745337 10674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: No bootstrap required, opened a new log
I20260806 06:46:23.746641 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10604
I20260806 06:46:23.746752 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:23.746938 6373 external_mini_cluster.cc:1428] Setting key 86f2d787d2e0c1fbb51d50e4c7a617c9
I20260806 06:46:23.747505 10674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } }
I20260806 06:46:23.747695 10674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:23.747763 10674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d4a38bd01a4944b616b512e9a5e17a, State: Initialized, Role: FOLLOWER
I20260806 06:46:23.747900 10674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } }
I20260806 06:46:23.747987 10674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:23.748075 10674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:23.748139 10674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:23.748898 10674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } }
I20260806 06:46:23.749013 10674 leader_election.cc:302] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75d4a38bd01a4944b616b512e9a5e17a; no voters:
I20260806 06:46:23.749198 10674 leader_election.cc:288] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:23.749234 10679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:23.749377 10679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [term 1 LEADER]: Becoming Leader. State: Replica: 75d4a38bd01a4944b616b512e9a5e17a, State: Running, Role: LEADER
I20260806 06:46:23.749521 10674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:23.749542 10679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } }
I20260806 06:46:23.750000 10679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75d4a38bd01a4944b616b512e9a5e17a. Latest consensus state: current_term: 1 leader_uuid: "75d4a38bd01a4944b616b512e9a5e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } } }
I20260806 06:46:23.750324 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75d4a38bd01a4944b616b512e9a5e17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75d4a38bd01a4944b616b512e9a5e17a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41589 } } }
I20260806 06:46:23.750494 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:23.750416 10679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a [sys.catalog]: This master's current role is: LEADER
W20260806 06:46:23.751603 10695 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:23.751713 10695 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:23.751823 10689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:23.751999 10689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:46:23 dist-test-slave-5l3k krb5kdc[10591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:23.753672 10689 catalog_manager.cc:1347] Generated new cluster ID: 68e23b69c3094977a70a875c342d320d
I20260806 06:46:23.753724 10689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:23.757380 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:23.747450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48492 (local address 127.6.57.126:41589)
0806 06:46:23.747625 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:23.747633 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:23.747664 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:46:23.747801 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:23.747809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:23.747861 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:23.747973 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:23.748801 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.750015 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.751144 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.751600 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.751719 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:23.754120 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:23.754124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:23.754125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:23.754136 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:23.755791 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:23.756178 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:23.756182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:23.756182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:23.756291 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:23.756491 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:23.756492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:23.756493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:23.756707 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:23.756860 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:23.756954 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 06:46:23.757087 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260806 06:46:23.768486 10689 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:23.769023 10689 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:23.772027 10689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: Generated new TSK 0
I20260806 06:46:23.772202 10689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:23.803701 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:23.914417 10701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:23.914676 10701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:23.914701 10701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:23.915834 10701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:23.915874 10701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:23.915900 10701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:23.917336 10701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:23.917568 10701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:23.917824 10701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:23.922590 10709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:23.922595 10706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:23.922617 10707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:23.923030 10701 server_base.cc:1034] running on GCE node
I20260806 06:46:23.923293 10701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:23.923648 10701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:23.924846 10701 hybrid_clock.cc:648] HybridClock initialized: now 1785998783924824 us; error 44 us; skew 500 ppm
Aug 06 06:46:23 dist-test-slave-5l3k krb5kdc[10591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998783, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:23.928151 10701 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:23.928580 10701 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:23.928655 10701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:23.928802 10701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:23.930315 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "4225a26c38784c74a1b21b538aa7a762"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "99faea7019623e1bbc83406ca0002b47"
server_key_iv: "e08edf08717a29da446f42814d3a1297"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.930800 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "4225a26c38784c74a1b21b538aa7a762"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "99faea7019623e1bbc83406ca0002b47"
server_key_iv: "e08edf08717a29da446f42814d3a1297"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.932735 10701 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:23.933636 10716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:23.933825 10701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:23.933931 10701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "4225a26c38784c74a1b21b538aa7a762"
format_stamp: "Formatted at 2026-08-06 06:46:23 on dist-test-slave-5l3k"
server_key: "99faea7019623e1bbc83406ca0002b47"
server_key_iv: "e08edf08717a29da446f42814d3a1297"
server_key_version: "encryptionkey@0"
I20260806 06:46:23.934012 10701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:23.946414 10701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:23.948622 10701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:23.948796 10701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:23.949035 10701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:23.949368 10701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:23.949419 10701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:23.949466 10701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:23.949512 10701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:23.956632 10701 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39717
I20260806 06:46:23.956668 10832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39717 every 8 connection(s)
I20260806 06:46:23.956955 10701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:23.959204 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10701
I20260806 06:46:23.959334 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:23.959546 6373 external_mini_cluster.cc:1428] Setting key b3d0c05a3348143196a96a468a2a016d
Aug 06 06:46:23 dist-test-slave-5l3k krb5kdc[10591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998783, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:23.968705 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:23.957716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:33985 (local address 127.6.57.126:41589)
0806 06:46:23.957776 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:23.957779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:23.958026 (+ 247us) server_negotiation.cc:407] Connection header received
0806 06:46:23.958516 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:23.958517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:23.958532 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:23.958589 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:23.959746 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.960453 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.961985 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.962353 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.962462 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:23.965214 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:23.965215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:23.965215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:23.965222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:23.966980 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:23.967479 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:23.967481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:23.967481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:23.967564 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:23.967885 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:23.967887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:23.967887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:23.968044 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:23.968136 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:23.968479 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 06:46:23.968592 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:46:23.968739 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:23.957750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41589 (local address 127.6.57.65:33985)
0806 06:46:23.957909 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:23.957943 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:23.958399 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:23.958661 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:23.958664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:23.958898 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:23.959635 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:23.959642 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.960547 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.960548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:23.961848 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:23.961852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:23.962476 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:23.962477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:23.962641 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:23.963191 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:23.963199 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:23.965083 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:23.967088 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:23.967096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:23.967099 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:23.967375 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:23.967674 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:23.967675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:23.967676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:23.967800 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:23.968152 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:23.968159 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:23.968326 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:46:23.968432 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 06:46:23.968570 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260806 06:46:23.969475 10833 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41589
I20260806 06:46:23.969594 10833 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:23.969745 10833 heartbeater.cc:511] Master 127.6.57.126:41589 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:46:23.970345 10638 ts_manager.cc:194] Registered new tserver with Master: 4225a26c38784c74a1b21b538aa7a762 (127.6.57.65:39717)
I20260806 06:46:23.971071 10638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:33985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:23.988832 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:24.069124 10838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:24.069301 10838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:24.069324 10838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:24.070457 10838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:24.070514 10838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:24.070559 10838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:24.072038 10838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:24.072274 10838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:24.072495 10838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:24.077348 10847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:24.077365 10845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:24.077368 10844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.077695 10838 server_base.cc:1034] running on GCE node
I20260806 06:46:24.078045 10838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:24.078274 10838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:24.079437 10838 hybrid_clock.cc:648] HybridClock initialized: now 1785998784079410 us; error 49 us; skew 500 ppm
Aug 06 06:46:24 dist-test-slave-5l3k krb5kdc[10591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998784, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:24.082299 10838 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:24.082654 10838 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:24.082702 10838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:24.082787 10838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:24.083700 10838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "62278da9e88948d8b7af0fd11088d3ec"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "aabec32f51c9b4478a71e5d318426e7a"
server_key_iv: "01513a688272c41a1c355e2589abd683"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.084002 10838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "62278da9e88948d8b7af0fd11088d3ec"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "aabec32f51c9b4478a71e5d318426e7a"
server_key_iv: "01513a688272c41a1c355e2589abd683"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.085237 10838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:24.085878 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.086052 10838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:24.086145 10838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "62278da9e88948d8b7af0fd11088d3ec"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "aabec32f51c9b4478a71e5d318426e7a"
server_key_iv: "01513a688272c41a1c355e2589abd683"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.086225 10838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:24.093031 10838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:24.094794 10838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:24.094942 10838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:24.095144 10838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:24.095438 10838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:24.095489 10838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.095538 10838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:24.095566 10838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.101151 10838 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:40783
I20260806 06:46:24.101214 10969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:40783 every 8 connection(s)
I20260806 06:46:24.101461 10838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:24.103237 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10838
I20260806 06:46:24.103345 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:24.103556 6373 external_mini_cluster.cc:1428] Setting key 8094e9057be39e6da05bcff932684450
Aug 06 06:46:24 dist-test-slave-5l3k krb5kdc[10591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998784, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:24.111290 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.102103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:50545 (local address 127.6.57.126:41589)
0806 06:46:24.102147 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.102149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.102370 (+ 221us) server_negotiation.cc:407] Connection header received
0806 06:46:24.102931 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.102933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.102946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.103014 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:24.104264 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.104751 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.106247 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.106451 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.106555 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.108575 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:24.108576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:24.108577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:24.108584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:24.109922 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.110314 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.110315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.110315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.110374 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.110649 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.110650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.110650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.110759 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:24.110829 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.111092 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.111191 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:24.111428 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.102073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41589 (local address 127.6.57.66:50545)
0806 06:46:24.102255 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.102288 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.102812 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.103103 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.103107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.103487 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:24.104164 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.104170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.104842 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.104843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.106137 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.106145 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.106583 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.106585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.106707 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.107171 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:24.107184 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:24.108466 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:24.109998 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:24.110004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:24.110007 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:24.110238 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:24.110442 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:24.110443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:24.110444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:24.110585 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:24.110830 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:24.110837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:24.110966 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.111067 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.111212 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260806 06:46:24.111923 10970 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41589
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
I20260806 06:46:24.112015 10970 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:24.112169 10970 heartbeater.cc:511] Master 127.6.57.126:41589 requested a full tablet report, sending...
I20260806 06:46:24.112532 10638 ts_manager.cc:194] Registered new tserver with Master: 62278da9e88948d8b7af0fd11088d3ec (127.6.57.66:40783)
I20260806 06:46:24.112936 10638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:50545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:24.128720 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:24.209149 10975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:24.209326 10975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:24.209347 10975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:24.210362 10975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:24.210392 10975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:24.210417 10975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:24.211866 10975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:24.212065 10975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:24.212244 10975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:24.217140 10980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.217263 10975 server_base.cc:1034] running on GCE node
W20260806 06:46:24.217137 10983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:24.217140 10981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.217607 10975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:24.217836 10975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:24.219007 10975 hybrid_clock.cc:648] HybridClock initialized: now 1785998784218994 us; error 30 us; skew 500 ppm
Aug 06 06:46:24 dist-test-slave-5l3k krb5kdc[10591](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998784, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:24.221340 10975 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:24.221706 10975 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:24.221772 10975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:24.221905 10975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:24.222980 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "4198425b51484733ae48e47d3f8c962c"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "feafbcd1b79edb2d04bc35d8f4a2a885"
server_key_iv: "1cbc2d40bfa6ec57786970a989166d3c"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.223331 10975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "4198425b51484733ae48e47d3f8c962c"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "feafbcd1b79edb2d04bc35d8f4a2a885"
server_key_iv: "1cbc2d40bfa6ec57786970a989166d3c"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.224697 10975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:24.225414 10989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.225623 10975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:24.225723 10975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "4198425b51484733ae48e47d3f8c962c"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "feafbcd1b79edb2d04bc35d8f4a2a885"
server_key_iv: "1cbc2d40bfa6ec57786970a989166d3c"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.225798 10975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:24.236503 10975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:24.238901 10975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:24.239065 10975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:24.239284 10975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:24.239583 10975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:24.239634 10975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.239666 10975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:24.239693 10975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.245673 10975 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:43899
I20260806 06:46:24.245713 11105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:43899 every 8 connection(s)
I20260806 06:46:24.245983 10975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:46:24 dist-test-slave-5l3k krb5kdc[10591](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998784, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:24.253297 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 10975
I20260806 06:46:24.253410 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:24.253644 6373 external_mini_cluster.cc:1428] Setting key d48596fb9db4f1072e961ff2de8882af
I20260806 06:46:24.255757 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.246774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34753 (local address 127.6.57.126:41589)
0806 06:46:24.246824 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.246826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.247039 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:46:24.247547 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.247548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.247558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.247628 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:24.248521 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.249024 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.250225 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.250548 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.250603 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.252832 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:24.252833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:24.252834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:24.252841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:24.254575 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.255020 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.255021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.255021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.255076 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.255267 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.255267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.255267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.255367 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:24.255417 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.255634 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.255694 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:24.255913 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.246745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41589 (local address 127.6.57.67:34753)
0806 06:46:24.246909 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.246950 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.247439 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.247669 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.247672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.247932 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:24.248428 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.248434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.249099 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.249100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.250122 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.250125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.250629 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.250630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.250754 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.251345 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:24.251354 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:24.252725 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:24.254647 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:24.254653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:24.254656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:24.254950 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:24.255124 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:24.255124 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:24.255124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:24.255217 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:24.255409 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:24.255412 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:24.255524 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.255605 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.255723 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260806 06:46:24.256388 11106 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41589
I20260806 06:46:24.256494 11106 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:24.256657 11106 heartbeater.cc:511] Master 127.6.57.126:41589 requested a full tablet report, sending...
I20260806 06:46:24.257009 10638 ts_manager.cc:194] Registered new tserver with Master: 4198425b51484733ae48e47d3f8c962c (127.6.57.67:43899)
I20260806 06:46:24.257396 10638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:34753
I20260806 06:46:24.265532 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:46:24.272468 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.266793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48494 (local address 127.6.57.126:41589)
0806 06:46:24.266839 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.266841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.266932 (+ 91us) server_negotiation.cc:407] Connection header received
0806 06:46:24.266966 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.266967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.266974 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.267020 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:24.267521 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.268003 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.268710 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.268875 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.268932 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.270012 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:24.270013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:24.270013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:24.270019 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:24.271395 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.271683 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.271684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.271684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.271731 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:24.272022 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:24.272022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:24.272023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:24.272153 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:24.272226 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.272336 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.272412 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:24.273900 10638 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:24.275410 10638 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:24.281622 11117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.277695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43899 (local address 127.0.0.1:37662)
0806 06:46:24.277815 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.277858 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.277944 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.278567 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.278571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.278575 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:24.278640 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.278642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.279566 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.279567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.280518 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.280521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.281401 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.281402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.281489 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.281500 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.281535 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.281568 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":65,"threads_started":1}
I20260806 06:46:24.281951 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.277980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.6.57.67:43899)
0806 06:46:24.278352 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.278356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.278366 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:24.278405 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.278407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.278424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.278722 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:24.278791 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.279470 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.280628 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.281306 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.281744 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.281815 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.281843 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.281878 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":88,"threads_started":1}
I20260806 06:46:24.282316 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.277721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39717 (local address 127.0.0.1:59406)
0806 06:46:24.278291 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.278301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.278351 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.278730 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.278731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.278733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:24.278767 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.278768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.280198 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.280199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.281189 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.281192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.282141 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.282142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.282197 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.282202 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.282246 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.282274 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":92,"threads_started":1}
I20260806 06:46:24.282388 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.277966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59406 (local address 127.6.57.65:39717)
0806 06:46:24.278115 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.278119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.278466 (+ 347us) server_negotiation.cc:407] Connection header received
0806 06:46:24.278533 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.278536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.278551 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.278635 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:24.278854 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.280079 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.281283 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.282044 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.282137 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.282196 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.282260 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.282305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260806 06:46:24.282680 11037 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1521702f904973b5ba77cf71a0a499 (DEFAULT_TABLE table=test-table [id=41966dc61ac640e785b0cfa0939b05cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:24.283051 11037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1521702f904973b5ba77cf71a0a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:24.283126 10767 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1521702f904973b5ba77cf71a0a499 (DEFAULT_TABLE table=test-table [id=41966dc61ac640e785b0cfa0939b05cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:24.283465 10767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1521702f904973b5ba77cf71a0a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:24.286723 11123 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Bootstrap starting.
I20260806 06:46:24.287231 11124 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Bootstrap starting.
I20260806 06:46:24.287288 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.277702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:40783 (local address 127.0.0.1:53950)
0806 06:46:24.278159 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.278169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.278208 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.278621 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.278623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.278625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:24.278686 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.278688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.280194 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.280196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.281291 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.281295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.287038 (+ 5743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.287040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.287115 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.287119 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.287185 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.287223 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":39,"thread_start_us":123,"threads_started":1}
I20260806 06:46:24.287528 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.278180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53950 (local address 127.6.57.66:40783)
0806 06:46:24.278380 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.278385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.278397 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:24.278440 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.278442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.278459 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.278563 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:24.278861 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.280061 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.281394 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.286894 (+ 5500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.287021 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.287092 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.287402 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.287448 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260806 06:46:24.287876 11123 tablet_bootstrap.cc:654] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:24.288067 11124 tablet_bootstrap.cc:654] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:24.288182 10902 tablet_service.cc:1467] Processing CreateTablet for tablet 4a1521702f904973b5ba77cf71a0a499 (DEFAULT_TABLE table=test-table [id=41966dc61ac640e785b0cfa0939b05cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:24.288569 10902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a1521702f904973b5ba77cf71a0a499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:24.288553 11123 log.cc:824] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:24.288780 11124 log.cc:824] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:24.289386 11123 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: No bootstrap required, opened a new log
I20260806 06:46:24.289471 11123 ts_tablet_manager.cc:1397] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:46:24.289611 11124 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: No bootstrap required, opened a new log
I20260806 06:46:24.289680 11124 ts_tablet_manager.cc:1397] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:46:24.291532 11127 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Bootstrap starting.
I20260806 06:46:24.291687 11123 raft_consensus.cc:348] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.291869 11123 raft_consensus.cc:374] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:24.291918 11123 raft_consensus.cc:729] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4198425b51484733ae48e47d3f8c962c, State: Initialized, Role: FOLLOWER
I20260806 06:46:24.291934 11124 raft_consensus.cc:348] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.292042 11123 consensus_queue.cc:260] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.292095 11124 raft_consensus.cc:374] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:24.292143 11124 raft_consensus.cc:729] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4225a26c38784c74a1b21b538aa7a762, State: Initialized, Role: FOLLOWER
I20260806 06:46:24.292160 11127 tablet_bootstrap.cc:654] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:24.292282 11124 consensus_queue.cc:260] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.292333 11123 ts_tablet_manager.cc:1428] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:46:24.292479 11106 heartbeater.cc:503] Master 127.6.57.126:41589 was elected leader, sending a full tablet report...
I20260806 06:46:24.292605 11124 ts_tablet_manager.cc:1428] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:24.292680 10833 heartbeater.cc:503] Master 127.6.57.126:41589 was elected leader, sending a full tablet report...
I20260806 06:46:24.292745 11127 log.cc:824] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:24.293838 11127 tablet_bootstrap.cc:492] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: No bootstrap required, opened a new log
I20260806 06:46:24.293918 11127 ts_tablet_manager.cc:1397] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:46:24.295291 11127 raft_consensus.cc:348] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.295428 11127 raft_consensus.cc:374] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:24.295473 11127 raft_consensus.cc:729] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62278da9e88948d8b7af0fd11088d3ec, State: Initialized, Role: FOLLOWER
I20260806 06:46:24.295583 11127 consensus_queue.cc:260] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.295867 11127 ts_tablet_manager.cc:1428] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:46:24.295873 10970 heartbeater.cc:503] Master 127.6.57.126:41589 was elected leader, sending a full tablet report...
W20260806 06:46:24.351960 10971 tablet.cc:2369] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:24.457744 10834 tablet.cc:2369] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:24.496587 11107 tablet.cc:2369] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:24.600180 11129 raft_consensus.cc:482] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:24.600310 11129 raft_consensus.cc:504] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.600719 11129 leader_election.cc:288] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4198425b51484733ae48e47d3f8c962c (127.6.57.67:43899), 62278da9e88948d8b7af0fd11088d3ec (127.6.57.66:40783)
I20260806 06:46:24.603588 11122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.601022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:47395 (local address 127.6.57.66:40783)
0806 06:46:24.601068 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.601070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.601108 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:46:24.601212 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.601213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.601225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.601296 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:24.601427 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.601886 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.602927 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.603289 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.603362 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.603373 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.603495 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.603529 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:46:24.603605 11132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.600911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:40783 (local address 127.6.57.65:47395)
0806 06:46:24.601044 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.601057 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.601131 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.601299 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.601300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.601302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:24.601356 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.601358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.601966 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.601967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.602836 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.602839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.603364 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.603365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.603420 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.603423 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.603486 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.603524 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":27,"thread_start_us":57,"threads_started":1}
I20260806 06:46:24.603678 11121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.601016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:46019 (local address 127.6.57.67:43899)
0806 06:46:24.601103 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.601105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.601137 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:46:24.601224 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.601225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.601236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.601303 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:24.601458 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.601896 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.602974 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.603341 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.603424 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.603433 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.603579 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.603612 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 06:46:24.603724 11133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.600920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43899 (local address 127.6.57.65:46019)
0806 06:46:24.601060 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.601071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.601150 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.601307 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.601308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.601310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:24.601386 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.601387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.601983 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.601984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.602885 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:24.602887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.603428 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.603429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:24.603497 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.603500 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.603571 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.603609 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260806 06:46:24.604051 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1521702f904973b5ba77cf71a0a499" candidate_uuid: "4225a26c38784c74a1b21b538aa7a762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4198425b51484733ae48e47d3f8c962c" is_pre_election: true
I20260806 06:46:24.604090 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1521702f904973b5ba77cf71a0a499" candidate_uuid: "4225a26c38784c74a1b21b538aa7a762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62278da9e88948d8b7af0fd11088d3ec" is_pre_election: true
I20260806 06:46:24.604218 11058 raft_consensus.cc:2454] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4225a26c38784c74a1b21b538aa7a762 in term 0.
I20260806 06:46:24.604218 10923 raft_consensus.cc:2454] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4225a26c38784c74a1b21b538aa7a762 in term 0.
I20260806 06:46:24.604453 10717 leader_election.cc:302] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4225a26c38784c74a1b21b538aa7a762, 62278da9e88948d8b7af0fd11088d3ec; no voters:
I20260806 06:46:24.604578 11129 raft_consensus.cc:2781] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:24.604643 11129 raft_consensus.cc:482] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:24.604667 11129 raft_consensus.cc:3037] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:24.605306 11129 raft_consensus.cc:504] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.605433 11129 leader_election.cc:288] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [CANDIDATE]: Term 1 election: Requested vote from peers 4198425b51484733ae48e47d3f8c962c (127.6.57.67:43899), 62278da9e88948d8b7af0fd11088d3ec (127.6.57.66:40783)
I20260806 06:46:24.605600 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1521702f904973b5ba77cf71a0a499" candidate_uuid: "4225a26c38784c74a1b21b538aa7a762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4198425b51484733ae48e47d3f8c962c"
I20260806 06:46:24.605624 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a1521702f904973b5ba77cf71a0a499" candidate_uuid: "4225a26c38784c74a1b21b538aa7a762" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62278da9e88948d8b7af0fd11088d3ec"
I20260806 06:46:24.605686 10923 raft_consensus.cc:3037] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:24.605691 11058 raft_consensus.cc:3037] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:24.606345 10923 raft_consensus.cc:2454] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4225a26c38784c74a1b21b538aa7a762 in term 1.
I20260806 06:46:24.606463 11058 raft_consensus.cc:2454] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4225a26c38784c74a1b21b538aa7a762 in term 1.
I20260806 06:46:24.606527 10717 leader_election.cc:302] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4225a26c38784c74a1b21b538aa7a762, 62278da9e88948d8b7af0fd11088d3ec; no voters:
I20260806 06:46:24.606657 11129 raft_consensus.cc:2781] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:24.606806 11129 raft_consensus.cc:686] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 1 LEADER]: Becoming Leader. State: Replica: 4225a26c38784c74a1b21b538aa7a762, State: Running, Role: LEADER
I20260806 06:46:24.606910 11129 consensus_queue.cc:237] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } }
I20260806 06:46:24.607805 10637 catalog_manager.cc:5495] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4225a26c38784c74a1b21b538aa7a762 (127.6.57.65). New cstate: current_term: 1 leader_uuid: "4225a26c38784c74a1b21b538aa7a762" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4225a26c38784c74a1b21b538aa7a762" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39717 } health_report { overall_health: HEALTHY } } }
I20260806 06:46:24.637552 11120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.635145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.6.57.65:39717)
0806 06:46:24.635207 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.635209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.635218 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:24.635304 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.635305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.635326 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.635386 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:24.635518 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.635962 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.636747 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:24.636928 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:24.636982 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:24.637170 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:24.637332 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:24.637395 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.637434 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.637452 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:46:24.639595 11058 raft_consensus.cc:1261] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 1 FOLLOWER]: Refusing update from remote peer 4225a26c38784c74a1b21b538aa7a762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:24.639595 10923 raft_consensus.cc:1261] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 1 FOLLOWER]: Refusing update from remote peer 4225a26c38784c74a1b21b538aa7a762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:24.639981 11134 consensus_queue.cc:1035] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62278da9e88948d8b7af0fd11088d3ec" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 40783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:24.640134 11129 consensus_queue.cc:1035] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4198425b51484733ae48e47d3f8c962c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:24.642598 11137 mvcc.cc:205] Tried to move back new op lower bound from 7315451021880369152 to 7315451021750800384. Current Snapshot: MvccSnapshot[applied={T|T < 7315451021880369152}]
I20260806 06:46:24.642875 11139 mvcc.cc:205] Tried to move back new op lower bound from 7315451021880369152 to 7315451021750800384. Current Snapshot: MvccSnapshot[applied={T|T < 7315451021880369152}]
I20260806 06:46:24.642994 11138 mvcc.cc:205] Tried to move back new op lower bound from 7315451021880369152 to 7315451021750800384. Current Snapshot: MvccSnapshot[applied={T|T < 7315451021880369152}]
I20260806 06:46:24.644894 10637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:24.644999 10637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:24.645761 10637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75d4a38bd01a4944b616b512e9a5e17a: Sending DeleteTablet for 3 replicas of tablet 4a1521702f904973b5ba77cf71a0a499
I20260806 06:46:24.646188 11037 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1521702f904973b5ba77cf71a0a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:24 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37662
I20260806 06:46:24.646284 10767 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1521702f904973b5ba77cf71a0a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:24 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:59406
I20260806 06:46:24.646515 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10701
I20260806 06:46:24.646540 11147 tablet_replica.cc:330] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762: stopping tablet replica
I20260806 06:46:24.646631 11147 raft_consensus.cc:2229] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:24.646718 11147 raft_consensus.cc:2258] T 4a1521702f904973b5ba77cf71a0a499 P 4225a26c38784c74a1b21b538aa7a762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:24.647126 11146 tablet_replica.cc:330] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: stopping tablet replica
I20260806 06:46:24.647244 11146 raft_consensus.cc:2229] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:24.647352 11146 raft_consensus.cc:2258] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:24.648017 10902 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a1521702f904973b5ba77cf71a0a499 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:24 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53950
I20260806 06:46:24.648083 11146 ts_tablet_manager.cc:1905] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:24.648243 11148 tablet_replica.cc:330] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: stopping tablet replica
I20260806 06:46:24.648422 11148 raft_consensus.cc:2229] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:24.648610 11148 raft_consensus.cc:2258] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:24.649163 11148 ts_tablet_manager.cc:1905] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:24.650218 11146 ts_tablet_manager.cc:1918] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:24.650336 11146 log.cc:1196] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/4a1521702f904973b5ba77cf71a0a499
I20260806 06:46:24.650665 11146 ts_tablet_manager.cc:1939] T 4a1521702f904973b5ba77cf71a0a499 P 4198425b51484733ae48e47d3f8c962c: Deleting consensus metadata
I20260806 06:46:24.651145 10623 catalog_manager.cc:4841] TS 4198425b51484733ae48e47d3f8c962c (127.6.57.67:43899): tablet 4a1521702f904973b5ba77cf71a0a499 (table test-table [id=41966dc61ac640e785b0cfa0939b05cd]) successfully deleted
I20260806 06:46:24.651553 11148 ts_tablet_manager.cc:1918] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:24.651640 11148 log.cc:1196] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/4a1521702f904973b5ba77cf71a0a499
I20260806 06:46:24.651909 11148 ts_tablet_manager.cc:1939] T 4a1521702f904973b5ba77cf71a0a499 P 62278da9e88948d8b7af0fd11088d3ec: Deleting consensus metadata
I20260806 06:46:24.652443 10621 catalog_manager.cc:4841] TS 62278da9e88948d8b7af0fd11088d3ec (127.6.57.66:40783): tablet 4a1521702f904973b5ba77cf71a0a499 (table test-table [id=41966dc61ac640e785b0cfa0939b05cd]) successfully deleted
W20260806 06:46:24.655354 10622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:39717 (error 108)
I20260806 06:46:24.655650 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10838
I20260806 06:46:24.656522 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.655798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39717 (local address 127.0.0.1:59418)
0806 06:46:24.656441 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.656484 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620}
W20260806 06:46:24.656656 10622 catalog_manager.cc:4584] TS 4225a26c38784c74a1b21b538aa7a762 (127.6.57.65:39717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a1521702f904973b5ba77cf71a0a499: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39717: connect: Connection refused (error 111)
I20260806 06:46:24.661760 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10975
I20260806 06:46:24.666980 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 10604
2026-08-06T06:46:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2209 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T06:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:24Z Disabled control of system clock
I20260806 06:46:24.696275 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45911
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:46257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:45911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:46:24.777779 11153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:24.777972 11153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:24.777994 11153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:24.779170 11153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:24.779212 11153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:24.779225 11153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:24.779237 11153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:24.779249 11153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:24.780629 11153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:45911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:24.780894 11153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:24.781188 11153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:24.786078 11159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:24.786031 11158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.786300 11153 server_base.cc:1034] running on GCE node
W20260806 06:46:24.786036 11161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.786641 11153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:24.786955 11153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:24.788165 11153 hybrid_clock.cc:648] HybridClock initialized: now 1785998784788150 us; error 33 us; skew 500 ppm
I20260806 06:46:24.789454 11153 webserver.cc:460] Webserver started at http://127.6.57.126:43679/ using document root <none> and password file <none>
I20260806 06:46:24.789701 11153 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:24.789772 11153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:24.789911 11153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:24.791005 11153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "95c0786b7a46440297b24f381f6196e1"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "395c7aab405fee387d780f137fc5e831"
server_key_iv: "2f2235ba923f4ee9be2dd7dc109213a6"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.791381 11153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "95c0786b7a46440297b24f381f6196e1"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "395c7aab405fee387d780f137fc5e831"
server_key_iv: "2f2235ba923f4ee9be2dd7dc109213a6"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.792656 11153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:24.793380 11167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.793565 11153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:24.793673 11153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "95c0786b7a46440297b24f381f6196e1"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "395c7aab405fee387d780f137fc5e831"
server_key_iv: "2f2235ba923f4ee9be2dd7dc109213a6"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.793748 11153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:24.796967 11153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:24.797228 11153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:24.797366 11153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:24.801245 11153 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:45911
I20260806 06:46:24.801301 11220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:45911 every 8 connection(s)
I20260806 06:46:24.801625 11153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:24.802317 11221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:24.804961 11221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: Bootstrap starting.
I20260806 06:46:24.805532 11221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:24.805821 11221 log.cc:824] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:24.806586 11221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: No bootstrap required, opened a new log
I20260806 06:46:24.808102 11221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } }
I20260806 06:46:24.808247 11221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:24.808269 11221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c0786b7a46440297b24f381f6196e1, State: Initialized, Role: FOLLOWER
I20260806 06:46:24.808429 11221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } }
I20260806 06:46:24.808490 11221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:24.808526 11221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:24.808574 11221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:24.809195 11221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } }
I20260806 06:46:24.809305 11221 leader_election.cc:302] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95c0786b7a46440297b24f381f6196e1; no voters:
I20260806 06:46:24.809490 11221 leader_election.cc:288] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:24.809571 11224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:24.809717 11221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:24.809752 11224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [term 1 LEADER]: Becoming Leader. State: Replica: 95c0786b7a46440297b24f381f6196e1, State: Running, Role: LEADER
I20260806 06:46:24.809856 11224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } }
I20260806 06:46:24.810245 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95c0786b7a46440297b24f381f6196e1. Latest consensus state: current_term: 1 leader_uuid: "95c0786b7a46440297b24f381f6196e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } } }
I20260806 06:46:24.810326 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:24.810448 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11153
I20260806 06:46:24.810468 11225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95c0786b7a46440297b24f381f6196e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c0786b7a46440297b24f381f6196e1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45911 } } }
I20260806 06:46:24.810561 11225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:24.810566 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:24.810678 11233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:24.810760 6373 external_mini_cluster.cc:1428] Setting key 137650816a75c4125752253955efc21b
I20260806 06:46:24.811595 11233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:24.813448 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.811234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37910 (local address 127.6.57.126:45911)
0806 06:46:24.811427 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.811437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.811470 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:46:24.811656 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.811664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.811709 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.811878 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:24.812987 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:24.812991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:24.812991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:24.812996 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:24.813036 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:24.813062 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:24.813126 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.813210 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.813276 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260806 06:46:24.814852 11233 catalog_manager.cc:1347] Generated new cluster ID: 2fec7a7f6d2248d0970ab218ffe14539
I20260806 06:46:24.814910 11233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:24.821408 11233 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:24.821556 11233 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:24.833693 11233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: Generated new TSK 0
I20260806 06:46:24.833890 11233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:24.841707 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--builtin_ntp_servers=127.6.57.84:46257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:46:24.926640 11245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:24.926931 11245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:24.926966 11245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:24.928099 11245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:24.928155 11245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:24.928195 11245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:24.929540 11245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:24.929792 11245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:24.930075 11245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:24.935765 11251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:24.935767 11253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.936036 11245 server_base.cc:1034] running on GCE node
W20260806 06:46:24.935786 11255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:24.936362 11245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:24.936668 11245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:24.937852 11245 hybrid_clock.cc:648] HybridClock initialized: now 1785998784937833 us; error 37 us; skew 500 ppm
I20260806 06:46:24.939257 11245 webserver.cc:460] Webserver started at http://127.6.57.65:34433/ using document root <none> and password file <none>
I20260806 06:46:24.939487 11245 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:24.939539 11245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:24.939658 11245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:24.940881 11245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "e7453ef610f2422989babf18ef58a69e"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "3f81b2cf4d6adfe57f2b9734a6b949fe"
server_key_iv: "93c143e1d9a6ddd7a6800665d9e67565"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.941239 11245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "e7453ef610f2422989babf18ef58a69e"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "3f81b2cf4d6adfe57f2b9734a6b949fe"
server_key_iv: "93c143e1d9a6ddd7a6800665d9e67565"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.942992 11245 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 06:46:24.943801 11261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.943969 11245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:24.944088 11245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "e7453ef610f2422989babf18ef58a69e"
format_stamp: "Formatted at 2026-08-06 06:46:24 on dist-test-slave-5l3k"
server_key: "3f81b2cf4d6adfe57f2b9734a6b949fe"
server_key_iv: "93c143e1d9a6ddd7a6800665d9e67565"
server_key_version: "encryptionkey@0"
I20260806 06:46:24.944162 11245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:24.947429 11245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:24.947670 11245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:24.947796 11245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:24.947994 11245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:24.948290 11245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:24.948320 11245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.948366 11245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:24.948390 11245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:24.953728 11245 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:34677
I20260806 06:46:24.953799 11378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:34677 every 8 connection(s)
I20260806 06:46:24.954099 11245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:24.956941 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11245
I20260806 06:46:24.957050 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:24.957103 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.954813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54827 (local address 127.6.57.126:45911)
0806 06:46:24.954867 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:24.954869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:24.955109 (+ 240us) server_negotiation.cc:407] Connection header received
0806 06:46:24.955556 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:24.955558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:24.955564 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:24.955606 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:24.956838 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:24.956840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:24.956840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:24.956845 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:24.956861 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:24.956866 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:24.956920 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:24.957009 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:46:24.957025 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:46:24.957255 6373 external_mini_cluster.cc:1428] Setting key 15ab98e56740f5cf5501bd1e8c9363d4
I20260806 06:46:24.957338 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:24.954812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45911 (local address 127.6.57.65:54827)
0806 06:46:24.954984 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:24.955014 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:24.955447 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:24.955677 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:24.955680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:24.955688 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:24.956607 (+ 919us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:24.956608 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:24.956651 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:24.956695 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:24.956924 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:24.956929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:24.956933 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:24.957018 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:46:24.957079 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260806 06:46:24.957825 11379 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45911
I20260806 06:46:24.957912 11379 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:24.958052 11379 heartbeater.cc:511] Master 127.6.57.126:45911 requested a full tablet report, sending...
I20260806 06:46:24.958268 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--builtin_ntp_servers=127.6.57.84:46257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 06:46:24.958590 11184 ts_manager.cc:194] Registered new tserver with Master: e7453ef610f2422989babf18ef58a69e (127.6.57.65:34677)
W20260806 06:46:25.039202 11382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:25.039384 11382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:25.039405 11382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:25.040544 11382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:25.040584 11382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:25.040611 11382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:25.041838 11382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:25.042014 11382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:25.042243 11382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:25.047144 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.047212 11388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.047253 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.047403 11382 server_base.cc:1034] running on GCE node
I20260806 06:46:25.047631 11382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:25.047887 11382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:25.049053 11382 hybrid_clock.cc:648] HybridClock initialized: now 1785998785049038 us; error 38 us; skew 500 ppm
I20260806 06:46:25.050189 11382 webserver.cc:460] Webserver started at http://127.6.57.66:42845/ using document root <none> and password file <none>
I20260806 06:46:25.050407 11382 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:25.050467 11382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:25.050617 11382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:25.051571 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "d0f80912b0f94513a6ca6f833f7c620b"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d9a6eac6193b5c2b3d6446bc263c918c"
server_key_iv: "536d1989a09305a0d683eb2174841c5b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.051954 11382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "d0f80912b0f94513a6ca6f833f7c620b"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d9a6eac6193b5c2b3d6446bc263c918c"
server_key_iv: "536d1989a09305a0d683eb2174841c5b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.053215 11382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:25.053828 11397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.053973 11382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:25.054064 11382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "d0f80912b0f94513a6ca6f833f7c620b"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d9a6eac6193b5c2b3d6446bc263c918c"
server_key_iv: "536d1989a09305a0d683eb2174841c5b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.054162 11382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:25.057344 11382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:25.057590 11382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:25.057735 11382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:25.057941 11382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:25.058233 11382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:25.058282 11382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.058320 11382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:25.058348 11382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.064090 11382 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:42961
I20260806 06:46:25.064162 11513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:42961 every 8 connection(s)
I20260806 06:46:25.064460 11382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:25.067090 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.065107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:51185 (local address 127.6.57.126:45911)
0806 06:46:25.065183 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.065185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.065421 (+ 236us) server_negotiation.cc:407] Connection header received
0806 06:46:25.065778 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.065779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.065784 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.065838 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.066891 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.066891 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.066892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.066895 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.066908 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.066912 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.066956 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.067005 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.067016 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:46:25.067267 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.065086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45911 (local address 127.6.57.66:51185)
0806 06:46:25.065270 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.065329 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.065704 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.065881 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.065884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.065892 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.066757 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.066758 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.066782 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.066807 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.066954 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.066956 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.066958 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.067003 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.067044 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260806 06:46:25.067695 11514 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45911
I20260806 06:46:25.067775 11514 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:25.067883 11514 heartbeater.cc:511] Master 127.6.57.126:45911 requested a full tablet report, sending...
I20260806 06:46:25.068197 11184 ts_manager.cc:194] Registered new tserver with Master: d0f80912b0f94513a6ca6f833f7c620b (127.6.57.66:42961)
I20260806 06:46:25.073451 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11382
I20260806 06:46:25.073527 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:25.073669 6373 external_mini_cluster.cc:1428] Setting key f38cc0ec33117601174e6c960c16bba6
I20260806 06:46:25.074525 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--builtin_ntp_servers=127.6.57.84:46257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 06:46:25.154573 11517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:25.154755 11517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:25.154778 11517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:25.155893 11517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:25.155930 11517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:25.155978 11517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:25.157336 11517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:25.157619 11517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:25.157928 11517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:25.162698 11525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.162739 11523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.162871 11522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.162880 11517 server_base.cc:1034] running on GCE node
I20260806 06:46:25.163110 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:25.163335 11517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:25.164489 11517 hybrid_clock.cc:648] HybridClock initialized: now 1785998785164474 us; error 27 us; skew 500 ppm
I20260806 06:46:25.165632 11517 webserver.cc:460] Webserver started at http://127.6.57.67:35689/ using document root <none> and password file <none>
I20260806 06:46:25.165841 11517 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:25.165899 11517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:25.166050 11517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:25.167102 11517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "0a17377d09fc45c68603e6e22e739208"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "5a8281ec50f88976ca1d98ab160818b7"
server_key_iv: "6ac12a16d21ce8de5f2a79901685eedd"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.167451 11517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "0a17377d09fc45c68603e6e22e739208"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "5a8281ec50f88976ca1d98ab160818b7"
server_key_iv: "6ac12a16d21ce8de5f2a79901685eedd"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.168743 11517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:25.169413 11531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.169576 11517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:25.169673 11517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "0a17377d09fc45c68603e6e22e739208"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "5a8281ec50f88976ca1d98ab160818b7"
server_key_iv: "6ac12a16d21ce8de5f2a79901685eedd"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.169739 11517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:25.172938 11517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:25.173179 11517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:25.173317 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:25.173511 11517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:25.173797 11517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:25.173844 11517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.173882 11517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:25.173908 11517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.179615 11517 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:46511
I20260806 06:46:25.179659 11646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:46511 every 8 connection(s)
I20260806 06:46:25.179945 11517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:25.182767 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.180642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:48109 (local address 127.6.57.126:45911)
0806 06:46:25.180737 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.180740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.180905 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:46:25.181376 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.181377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.181384 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.181442 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.182509 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.182510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.182511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.182516 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.182532 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.182536 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.182586 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.182682 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.182699 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:46:25.182922 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.180601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45911 (local address 127.6.57.67:48109)
0806 06:46:25.180801 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.180830 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.181270 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.181502 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.181506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.181513 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.182334 (+ 821us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.182335 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.182360 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.182384 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.182591 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.182596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.182611 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.182690 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.182744 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260806 06:46:25.183328 11647 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45911
I20260806 06:46:25.183414 11647 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:25.183540 11647 heartbeater.cc:511] Master 127.6.57.126:45911 requested a full tablet report, sending...
I20260806 06:46:25.183862 11184 ts_manager.cc:194] Registered new tserver with Master: 0a17377d09fc45c68603e6e22e739208 (127.6.57.67:46511)
I20260806 06:46:25.189409 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11517
I20260806 06:46:25.189533 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:25.189723 6373 external_mini_cluster.cc:1428] Setting key 70a8abc67ad2a35ce037b2813c22329d
I20260806 06:46:25.190719 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:46:25.193123 11240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.192082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.6.57.126:45911)
0806 06:46:25.192167 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.192169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.192266 (+ 97us) server_negotiation.cc:407] Connection header received
0806 06:46:25.192348 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.192349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.192355 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.192404 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.192891 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.192893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.192893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.192898 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.192912 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.192917 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.192972 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.193030 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.193045 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 06:46:25.194721 11184 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:25.196319 11184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:25.200166 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:46511 (local address 127.0.0.1:43022)
0806 06:46:25.198664 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.198703 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.198761 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.199265 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.199266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.199269 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.199698 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.199699 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.199711 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.199737 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.199999 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.200005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.200007 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.200079 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.200094 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"spinlock_wait_cycles":768,"thread_start_us":185,"threads_started":1}
I20260806 06:46:25.200304 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.6.57.66:42961)
0806 06:46:25.199360 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.199366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.199376 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:25.199401 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.199403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.199412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.199501 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.199962 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.199968 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.199969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.199973 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.200006 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.200037 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.200122 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.200193 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.200209 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260806 06:46:25.200330 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.6.57.65:34677)
0806 06:46:25.198849 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.198854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.198866 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:25.198893 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.198895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.198905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.199007 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.199923 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.199932 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.199932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.199937 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.199986 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.200018 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.200077 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.200231 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.200248 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260806 06:46:25.200328 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.6.57.67:46511)
0806 06:46:25.199087 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.199091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.199101 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:25.199124 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.199126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.199156 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.199420 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.199859 (+ 439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.199863 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.199863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.199867 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.199919 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.199937 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.200007 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.200231 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.200249 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":79,"threads_started":1}
I20260806 06:46:25.200724 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34677 (local address 127.0.0.1:37144)
0806 06:46:25.198750 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.198759 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.198795 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.199020 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.199026 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.199032 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.199694 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.199694 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.199710 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.199733 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.200092 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.200093 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.200094 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.200144 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.200158 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260806 06:46:25.200747 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.198434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:42961 (local address 127.0.0.1:47256)
0806 06:46:25.198848 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.198856 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.198886 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.199502 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.199503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.199506 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.199858 (+ 352us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.199859 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.199862 (+ 3us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.199869 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.200130 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.200131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.200132 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.200183 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.200194 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":176,"threads_started":1}
I20260806 06:46:25.201012 11580 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1346a018463d918b14984fb290a7 (DEFAULT_TABLE table=test-table [id=5cd75b6a7d1f41ff9013fd071a317220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:25.201375 11580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1346a018463d918b14984fb290a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:25.201354 11447 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1346a018463d918b14984fb290a7 (DEFAULT_TABLE table=test-table [id=5cd75b6a7d1f41ff9013fd071a317220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:25.201335 11311 tablet_service.cc:1467] Processing CreateTablet for tablet fbba1346a018463d918b14984fb290a7 (DEFAULT_TABLE table=test-table [id=5cd75b6a7d1f41ff9013fd071a317220]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:25.201663 11447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1346a018463d918b14984fb290a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:25.201697 11311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbba1346a018463d918b14984fb290a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:25.204306 11661 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Bootstrap starting.
I20260806 06:46:25.204617 11662 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Bootstrap starting.
I20260806 06:46:25.204983 11663 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: Bootstrap starting.
I20260806 06:46:25.205067 11661 tablet_bootstrap.cc:654] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:25.205309 11662 tablet_bootstrap.cc:654] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:25.205468 11661 log.cc:824] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:25.205639 11662 log.cc:824] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:25.205724 11663 tablet_bootstrap.cc:654] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:25.206054 11663 log.cc:824] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:25.206364 11661 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: No bootstrap required, opened a new log
I20260806 06:46:25.206369 11662 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: No bootstrap required, opened a new log
I20260806 06:46:25.206441 11661 ts_tablet_manager.cc:1397] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:25.206452 11662 ts_tablet_manager.cc:1397] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:25.207048 11663 tablet_bootstrap.cc:492] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: No bootstrap required, opened a new log
I20260806 06:46:25.207185 11663 ts_tablet_manager.cc:1397] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:25.208621 11661 raft_consensus.cc:348] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.208779 11661 raft_consensus.cc:374] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:25.208844 11661 raft_consensus.cc:729] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f80912b0f94513a6ca6f833f7c620b, State: Initialized, Role: FOLLOWER
I20260806 06:46:25.208968 11661 consensus_queue.cc:260] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.209050 11662 raft_consensus.cc:348] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.209276 11514 heartbeater.cc:503] Master 127.6.57.126:45911 was elected leader, sending a full tablet report...
I20260806 06:46:25.209345 11662 raft_consensus.cc:374] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:25.209430 11662 raft_consensus.cc:729] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a17377d09fc45c68603e6e22e739208, State: Initialized, Role: FOLLOWER
I20260806 06:46:25.209250 11661 ts_tablet_manager.cc:1428] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:46:25.209556 11662 consensus_queue.cc:260] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.209815 11662 ts_tablet_manager.cc:1428] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:46:25.209832 11647 heartbeater.cc:503] Master 127.6.57.126:45911 was elected leader, sending a full tablet report...
I20260806 06:46:25.209853 11663 raft_consensus.cc:348] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.209987 11663 raft_consensus.cc:374] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:25.210016 11663 raft_consensus.cc:729] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7453ef610f2422989babf18ef58a69e, State: Initialized, Role: FOLLOWER
I20260806 06:46:25.210171 11663 consensus_queue.cc:260] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.210423 11663 ts_tablet_manager.cc:1428] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:25.210604 11379 heartbeater.cc:503] Master 127.6.57.126:45911 was elected leader, sending a full tablet report...
W20260806 06:46:25.315053 11515 tablet.cc:2369] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:25.361464 11668 raft_consensus.cc:482] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:25.361588 11668 raft_consensus.cc:504] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.361891 11668 leader_election.cc:288] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7453ef610f2422989babf18ef58a69e (127.6.57.65:34677), d0f80912b0f94513a6ca6f833f7c620b (127.6.57.66:42961)
I20260806 06:46:25.363180 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.362146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:46953 (local address 127.6.57.66:42961)
0806 06:46:25.362194 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.362196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.362207 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:25.362239 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.362240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.362247 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.362326 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.362913 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.362915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.362915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.362919 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.362939 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.362943 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.363016 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.363083 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.363099 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:25.363181 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.362076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:42961 (local address 127.6.57.67:46953)
0806 06:46:25.362128 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.362141 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.362182 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.362335 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.362336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.362339 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.362786 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.362786 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.362794 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.362805 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.363035 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.363036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.363037 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.363071 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.363082 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:46:25.363260 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.362154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:53035 (local address 127.6.57.65:34677)
0806 06:46:25.362196 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.362198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.362341 (+ 143us) server_negotiation.cc:407] Connection header received
0806 06:46:25.362379 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.362380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.362385 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.362432 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.362998 (+ 566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.363000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.363001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.363005 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.363050 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.363054 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.363113 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.363177 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.363190 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:25.363299 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.362076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34677 (local address 127.6.57.67:53035)
0806 06:46:25.362253 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.362262 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.362335 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.362452 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.362453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.362455 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:25.362904 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:25.362905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:25.362909 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:25.362917 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:25.363127 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:25.363127 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:25.363128 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 06:46:25.363170 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:46:25.363182 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":12,"thread_start_us":106,"threads_started":1}
I20260806 06:46:25.363595 11331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1346a018463d918b14984fb290a7" candidate_uuid: "0a17377d09fc45c68603e6e22e739208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7453ef610f2422989babf18ef58a69e" is_pre_election: true
I20260806 06:46:25.363595 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1346a018463d918b14984fb290a7" candidate_uuid: "0a17377d09fc45c68603e6e22e739208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f80912b0f94513a6ca6f833f7c620b" is_pre_election: true
I20260806 06:46:25.363751 11331 raft_consensus.cc:2454] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a17377d09fc45c68603e6e22e739208 in term 0.
I20260806 06:46:25.363751 11468 raft_consensus.cc:2454] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a17377d09fc45c68603e6e22e739208 in term 0.
I20260806 06:46:25.363972 11535 leader_election.cc:302] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a17377d09fc45c68603e6e22e739208, d0f80912b0f94513a6ca6f833f7c620b; no voters:
I20260806 06:46:25.364094 11668 raft_consensus.cc:2781] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:25.364174 11668 raft_consensus.cc:482] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:25.364197 11668 raft_consensus.cc:3037] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:25.364867 11668 raft_consensus.cc:504] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.364996 11668 leader_election.cc:288] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [CANDIDATE]: Term 1 election: Requested vote from peers e7453ef610f2422989babf18ef58a69e (127.6.57.65:34677), d0f80912b0f94513a6ca6f833f7c620b (127.6.57.66:42961)
I20260806 06:46:25.365195 11468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1346a018463d918b14984fb290a7" candidate_uuid: "0a17377d09fc45c68603e6e22e739208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f80912b0f94513a6ca6f833f7c620b"
I20260806 06:46:25.365286 11468 raft_consensus.cc:3037] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:25.365235 11331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbba1346a018463d918b14984fb290a7" candidate_uuid: "0a17377d09fc45c68603e6e22e739208" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7453ef610f2422989babf18ef58a69e"
I20260806 06:46:25.365376 11331 raft_consensus.cc:3037] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:25.366027 11468 raft_consensus.cc:2454] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a17377d09fc45c68603e6e22e739208 in term 1.
I20260806 06:46:25.366173 11535 leader_election.cc:302] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a17377d09fc45c68603e6e22e739208, d0f80912b0f94513a6ca6f833f7c620b; no voters:
I20260806 06:46:25.366248 11668 raft_consensus.cc:2781] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:25.366320 11331 raft_consensus.cc:2454] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a17377d09fc45c68603e6e22e739208 in term 1.
I20260806 06:46:25.366372 11668 raft_consensus.cc:686] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 1 LEADER]: Becoming Leader. State: Replica: 0a17377d09fc45c68603e6e22e739208, State: Running, Role: LEADER
I20260806 06:46:25.366456 11668 consensus_queue.cc:237] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } }
I20260806 06:46:25.367447 11184 catalog_manager.cc:5495] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a17377d09fc45c68603e6e22e739208 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "0a17377d09fc45c68603e6e22e739208" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a17377d09fc45c68603e6e22e739208" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:25.380556 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.379723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43024 (local address 127.6.57.67:46511)
0806 06:46:25.379773 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.379775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.379785 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:25.379870 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.379871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.379880 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.379931 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.380393 (+ 462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.380394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.380395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.380398 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.380410 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.380414 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.380460 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.380499 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.380509 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 06:46:25.382187 11331 raft_consensus.cc:1261] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 1 FOLLOWER]: Refusing update from remote peer 0a17377d09fc45c68603e6e22e739208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:25.382200 11468 raft_consensus.cc:1261] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 1 FOLLOWER]: Refusing update from remote peer 0a17377d09fc45c68603e6e22e739208: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:25.382542 11671 consensus_queue.cc:1035] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7453ef610f2422989babf18ef58a69e" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:25.382653 11668 consensus_queue.cc:1035] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f80912b0f94513a6ca6f833f7c620b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:25.387394 11184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:25.387467 11184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:25.388121 11184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95c0786b7a46440297b24f381f6196e1: Sending DeleteTablet for 3 replicas of tablet fbba1346a018463d918b14984fb290a7
I20260806 06:46:25.388423 11580 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1346a018463d918b14984fb290a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:25 UTC) from {username='slave'} at 127.0.0.1:43022
I20260806 06:46:25.388425 11447 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1346a018463d918b14984fb290a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:25 UTC) from {username='slave'} at 127.0.0.1:47256
I20260806 06:46:25.388641 11685 tablet_replica.cc:330] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: stopping tablet replica
I20260806 06:46:25.388651 11311 tablet_service.cc:1514] Processing DeleteTablet for tablet fbba1346a018463d918b14984fb290a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:25 UTC) from {username='slave'} at 127.0.0.1:37144
I20260806 06:46:25.388768 11685 raft_consensus.cc:2229] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:25.388823 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11245
I20260806 06:46:25.388863 11685 raft_consensus.cc:2258] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:25.388922 11687 tablet_replica.cc:330] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e: stopping tablet replica
I20260806 06:46:25.389089 11687 raft_consensus.cc:2229] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:25.389181 11687 raft_consensus.cc:2258] T fbba1346a018463d918b14984fb290a7 P e7453ef610f2422989babf18ef58a69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:25.389899 11685 ts_tablet_manager.cc:1905] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:25.391448 11685 ts_tablet_manager.cc:1918] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:25.391538 11685 log.cc:1196] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/fbba1346a018463d918b14984fb290a7
I20260806 06:46:25.391791 11685 ts_tablet_manager.cc:1939] T fbba1346a018463d918b14984fb290a7 P d0f80912b0f94513a6ca6f833f7c620b: Deleting consensus metadata
I20260806 06:46:25.392343 11171 catalog_manager.cc:4841] TS d0f80912b0f94513a6ca6f833f7c620b (127.6.57.66:42961): tablet fbba1346a018463d918b14984fb290a7 (table test-table [id=5cd75b6a7d1f41ff9013fd071a317220]) successfully deleted
I20260806 06:46:25.398605 11686 tablet_replica.cc:330] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: stopping tablet replica
I20260806 06:46:25.398715 11686 raft_consensus.cc:2229] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:25.398811 11686 raft_consensus.cc:2258] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 06:46:25.399205 11168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:34677 (error 108)
I20260806 06:46:25.399248 11686 ts_tablet_manager.cc:1905] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:25.399456 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11382
I20260806 06:46:25.400630 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.399626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34677 (local address 127.0.0.1:37146)
0806 06:46:25.400530 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.400582 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:34677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":870}
W20260806 06:46:25.400828 11168 catalog_manager.cc:4584] TS e7453ef610f2422989babf18ef58a69e (127.6.57.65:34677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbba1346a018463d918b14984fb290a7: Network error: Client connection negotiation failed: client connection to 127.6.57.65:34677: connect: Connection refused (error 111)
I20260806 06:46:25.401536 11686 ts_tablet_manager.cc:1918] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:25.401621 11686 log.cc:1196] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/fbba1346a018463d918b14984fb290a7
I20260806 06:46:25.401908 11686 ts_tablet_manager.cc:1939] T fbba1346a018463d918b14984fb290a7 P 0a17377d09fc45c68603e6e22e739208: Deleting consensus metadata
I20260806 06:46:25.402309 11170 catalog_manager.cc:4841] TS 0a17377d09fc45c68603e6e22e739208 (127.6.57.67:46511): tablet fbba1346a018463d918b14984fb290a7 (table test-table [id=5cd75b6a7d1f41ff9013fd071a317220]) successfully deleted
I20260806 06:46:25.405669 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11517
I20260806 06:46:25.412680 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11153
2026-08-06T06:46:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (744 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 06:46:25.432866 6373 mini_oidc.cc:152] Starting JWKS server
I20260806 06:46:25.433895 6373 webserver.cc:460] Webserver started at https://127.0.0.1:44487/ using document root <none> and password file <none>
I20260806 06:46:25.434020 6373 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:46:25.434211 6373 webserver.cc:460] Webserver started at http://127.0.0.1:36905/ using document root <none> and password file <none>
2026-08-06T06:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:25Z Disabled control of system clock
I20260806 06:46:25.442541 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41187
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:44869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41187
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44487/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/testchainca.cert with env {}
W20260806 06:46:25.523100 11694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:25.523335 11694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:25.523372 11694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:25.524545 11694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:25.524612 11694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:25.524636 11694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:25.524665 11694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:25.524689 11694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:46:25.524716 11694 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:46:25.524744 11694 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44487/jwks.json
I20260806 06:46:25.526278 11694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44487/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:25.526577 11694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:25.526795 11694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:25.531564 11700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.531553 11702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.531610 11694 server_base.cc:1034] running on GCE node
W20260806 06:46:25.531610 11699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.532092 11694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:25.532379 11694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:25.533540 11694 hybrid_clock.cc:648] HybridClock initialized: now 1785998785533533 us; error 33 us; skew 500 ppm
I20260806 06:46:25.534952 11694 webserver.cc:460] Webserver started at http://127.6.57.126:39473/ using document root <none> and password file <none>
I20260806 06:46:25.535238 11694 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:25.535310 11694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:25.535512 11694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:25.536705 11694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "bf9418d9bc68429d89dd8644f2c31ef4"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "010ef74cf1d4bc2ddaca3700a8aaa83b"
server_key_iv: "1bd3f64e62f405dfc6d0557279d7d55b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.537006 11694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "bf9418d9bc68429d89dd8644f2c31ef4"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "010ef74cf1d4bc2ddaca3700a8aaa83b"
server_key_iv: "1bd3f64e62f405dfc6d0557279d7d55b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.538199 11694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:25.538918 11710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.539091 11694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.539189 11694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "bf9418d9bc68429d89dd8644f2c31ef4"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "010ef74cf1d4bc2ddaca3700a8aaa83b"
server_key_iv: "1bd3f64e62f405dfc6d0557279d7d55b"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.539263 11694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:25.547365 11694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:25.547637 11694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:25.547783 11694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:25.551888 11694 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41187
I20260806 06:46:25.551918 11764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41187 every 8 connection(s)
I20260806 06:46:25.552230 11694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:25.553126 11765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:25.555445 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: Bootstrap starting.
I20260806 06:46:25.556033 11765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:25.556329 11765 log.cc:824] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:25.556821 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11694
I20260806 06:46:25.556931 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:25.557096 11765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: No bootstrap required, opened a new log
I20260806 06:46:25.557133 6373 external_mini_cluster.cc:1428] Setting key 2b24dd66dbfe9607f0e01d2a82808211
I20260806 06:46:25.559011 11765 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } }
I20260806 06:46:25.559144 11765 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:25.559167 11765 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9418d9bc68429d89dd8644f2c31ef4, State: Initialized, Role: FOLLOWER
I20260806 06:46:25.559309 11765 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } }
I20260806 06:46:25.559368 11765 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:25.559405 11765 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:25.559453 11765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:25.559967 11765 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } }
I20260806 06:46:25.560072 11765 leader_election.cc:302] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf9418d9bc68429d89dd8644f2c31ef4; no voters:
I20260806 06:46:25.560257 11765 leader_election.cc:288] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:25.560292 11770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:25.560458 11770 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [term 1 LEADER]: Becoming Leader. State: Replica: bf9418d9bc68429d89dd8644f2c31ef4, State: Running, Role: LEADER
I20260806 06:46:25.560523 11765 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:25.560568 11770 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } }
I20260806 06:46:25.560916 11770 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf9418d9bc68429d89dd8644f2c31ef4. Latest consensus state: current_term: 1 leader_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } } }
I20260806 06:46:25.560928 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9418d9bc68429d89dd8644f2c31ef4" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41187 } } }
I20260806 06:46:25.561060 11770 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [sys.catalog]: This master's current role is: LEADER
W20260806 06:46:25.562281 11784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:25.562326 11784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:25.562654 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:25.563059 11786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:25.563258 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.557606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.6.57.126:41187)
0806 06:46:25.557832 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.557844 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.557869 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:46:25.558315 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.558321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.558375 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.558553 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.558792 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.560113 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.561129 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.561566 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.561701 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.562404 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.562408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.562409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.562642 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.562685 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.562700 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.562778 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.562937 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.563013 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260806 06:46:25.563273 11786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:25.564186 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:46:25.565492 11786 catalog_manager.cc:1347] Generated new cluster ID: 8832ef9031b0427f93cf18258bdfbe47
I20260806 06:46:25.565552 11786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:25.591446 11786 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:25.591979 11786 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:25.594890 11786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf9418d9bc68429d89dd8644f2c31ef4: Generated new TSK 0
I20260806 06:46:25.595075 11786 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:46:25.626796 11795 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 06:46:25.627025 11795 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:46:25.627209 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.604510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.6.57.126:41187)
0806 06:46:25.604555 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.604558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.604699 (+ 141us) server_negotiation.cc:407] Connection header received
0806 06:46:25.604809 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.604810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.604825 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.604903 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:46:25.605063 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.605597 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.606379 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.606638 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.606708 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.606913 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:46:25.626791 (+ 19878us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:46:25.627061 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:46:25.627290 11769 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 06:46:25.627588 11794 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:25.604438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41187 (local address 127.0.0.1:46648)
0806 06:46:25.604621 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.604638 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.604724 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.604915 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.604916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.604918 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:46:25.604981 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:25.604984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.605692 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.605693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:25.606287 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:25.606289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.606712 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.606713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:25.606771 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.606797 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:46:25.626960 (+ 20163us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 06:46:25.627083 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.57.126:41187: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.57.126:41187: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44487/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 06:46:25.628120 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11694
2026-08-06T06:46:25Z chronyd exiting
I20260806 06:46:25.665733 6373 test_util.cc:175] -----------------------------------------------
I20260806 06:46:25.665841 6373 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785998763187424-6373-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 06:46:25.666723 6373 mini_oidc.cc:152] Starting JWKS server
I20260806 06:46:25.667408 6373 webserver.cc:460] Webserver started at https://127.0.0.1:45853/ using document root <none> and password file <none>
I20260806 06:46:25.667471 6373 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:46:25.667631 6373 webserver.cc:460] Webserver started at http://127.0.0.1:40287/ using document root <none> and password file <none>
2026-08-06T06:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:25Z Disabled control of system clock
I20260806 06:46:25.676384 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41545
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:44895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41545
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45853/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/testchainca.cert with env {}
W20260806 06:46:25.757763 11801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:25.758015 11801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:25.758055 11801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:25.759227 11801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:25.759287 11801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:25.759308 11801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:25.759328 11801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:25.759356 11801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:46:25.759377 11801 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:46:25.759405 11801 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45853/jwks.json
I20260806 06:46:25.760855 11801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45853/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:25.761145 11801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:25.761358 11801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:25.766088 11806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.766093 11809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.766132 11807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.766345 11801 server_base.cc:1034] running on GCE node
I20260806 06:46:25.766754 11801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:25.767043 11801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:25.768217 11801 hybrid_clock.cc:648] HybridClock initialized: now 1785998785768206 us; error 32 us; skew 500 ppm
I20260806 06:46:25.773334 11801 webserver.cc:460] Webserver started at http://127.6.57.126:38083/ using document root <none> and password file <none>
I20260806 06:46:25.773561 11801 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:25.773623 11801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:25.773797 11801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:25.774901 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "a3ea141eca6e40f593d7ae346f0c89f1"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "8e1fdc8c65bc3e157460fbec1fb0c2e5"
server_key_iv: "9eccb6d850e0794e5fda24b8650d6228"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.775260 11801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "a3ea141eca6e40f593d7ae346f0c89f1"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "8e1fdc8c65bc3e157460fbec1fb0c2e5"
server_key_iv: "9eccb6d850e0794e5fda24b8650d6228"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.776592 11801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:25.777328 11815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.777498 11801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:25.777598 11801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "a3ea141eca6e40f593d7ae346f0c89f1"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "8e1fdc8c65bc3e157460fbec1fb0c2e5"
server_key_iv: "9eccb6d850e0794e5fda24b8650d6228"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.777671 11801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:25.786195 11801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:25.786459 11801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:25.786674 11801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:25.790663 11801 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41545
I20260806 06:46:25.790706 11868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41545 every 8 connection(s)
I20260806 06:46:25.791031 11801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:25.791966 11869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:25.794243 11869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1: Bootstrap starting.
I20260806 06:46:25.794868 11869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:25.795148 11869 log.cc:824] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:25.795794 11869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1: No bootstrap required, opened a new log
I20260806 06:46:25.797429 11869 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } }
I20260806 06:46:25.797583 11869 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:25.797606 11869 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ea141eca6e40f593d7ae346f0c89f1, State: Initialized, Role: FOLLOWER
I20260806 06:46:25.797760 11869 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } }
I20260806 06:46:25.797823 11869 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:25.797861 11869 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:25.797909 11869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:25.798437 11869 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } }
I20260806 06:46:25.798604 11869 leader_election.cc:302] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3ea141eca6e40f593d7ae346f0c89f1; no voters:
I20260806 06:46:25.798777 11869 leader_election.cc:288] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:25.798813 11872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:25.798995 11872 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [term 1 LEADER]: Becoming Leader. State: Replica: a3ea141eca6e40f593d7ae346f0c89f1, State: Running, Role: LEADER
I20260806 06:46:25.799036 11869 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:25.799125 11872 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } }
I20260806 06:46:25.799530 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } } }
I20260806 06:46:25.799659 11874 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3ea141eca6e40f593d7ae346f0c89f1. Latest consensus state: current_term: 1 leader_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ea141eca6e40f593d7ae346f0c89f1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41545 } } }
I20260806 06:46:25.799747 11874 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:25.799690 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:25.800168 11881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:25.800665 11881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:25.800796 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11801
I20260806 06:46:25.800899 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:25.801079 6373 external_mini_cluster.cc:1428] Setting key a435f6a64f96143f5e4ad1c6359ae8cf
I20260806 06:46:25.802326 11881 catalog_manager.cc:1347] Generated new cluster ID: ecd40de73c96419391d8c575232ca1e4
I20260806 06:46:25.802395 11881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:25.810765 11881 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:25.811373 11881 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:25.812431 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.801494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.6.57.126:41545)
0806 06:46:25.801802 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.801811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.802082 (+ 271us) server_negotiation.cc:407] Connection header received
0806 06:46:25.802185 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.802190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.806551 (+ 4361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.806717 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:25.806981 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.808686 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.809887 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.810476 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.810652 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.811535 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:25.811539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:25.811539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:25.811745 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:25.811787 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:25.811801 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:25.811866 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:25.811974 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:46:25.812201 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":78,"threads_started":1}
I20260806 06:46:25.813338 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:46:25.820801 11881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3ea141eca6e40f593d7ae346f0c89f1: Generated new TSK 0
I20260806 06:46:25.820993 11881 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 06:46:25.842552 11899 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 06:46:25.842638 11899 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 06:46:25.842960 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:25.828595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52454 (local address 127.6.57.126:41545)
0806 06:46:25.828657 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:25.828659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:25.828782 (+ 123us) server_negotiation.cc:407] Connection header received
0806 06:46:25.828828 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:25.828829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:25.828843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:25.828930 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:46:25.829020 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.829785 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.830574 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.830895 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.830977 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.831102 (+ 125us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:46:25.842733 (+ 11631us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:46:25.842889 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:46:25.843003 11898 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:25.828529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41545 (local address 127.0.0.1:52454)
0806 06:46:25.828707 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:25.828723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:25.828775 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:25.828926 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:25.828927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:25.828928 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:46:25.828976 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:25.828977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.829861 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.829861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:25.830424 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:25.830426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:25.830983 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:25.830984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:25.831025 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:25.831029 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:46:25.842842 (+ 11813us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 06:46:25.842932 (+ 90us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.57.126:41545: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
W20260806 06:46:25.843034 11891 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52454: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 06:46:25.843389 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11801
2026-08-06T06:46:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 06:46:25.883174 6373 mini_oidc.cc:152] Starting JWKS server
I20260806 06:46:25.883854 6373 webserver.cc:460] Webserver started at https://127.0.0.1:42697/ using document root <none> and password file <none>
I20260806 06:46:25.883899 6373 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 06:46:25.884021 6373 webserver.cc:460] Webserver started at http://127.0.0.1:44223/ using document root <none> and password file <none>
2026-08-06T06:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:25Z Disabled control of system clock
I20260806 06:46:25.892935 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41955
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:44925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41955
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42697/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:46:25.975330 11905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:25.975530 11905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:25.975553 11905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:25.976688 11905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:25.976720 11905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:25.976733 11905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:25.976745 11905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:25.976758 11905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:46:25.976769 11905 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:46:25.976785 11905 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42697/jwks.json
I20260806 06:46:25.978096 11905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41955
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42697/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:25.978295 11905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:25.978557 11905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:25.983284 11910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.983282 11913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:25.983259 11911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:25.983569 11905 server_base.cc:1034] running on GCE node
I20260806 06:46:25.983825 11905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:25.984088 11905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:25.985270 11905 hybrid_clock.cc:648] HybridClock initialized: now 1785998785985253 us; error 40 us; skew 500 ppm
I20260806 06:46:25.986644 11905 webserver.cc:460] Webserver started at http://127.6.57.126:45667/ using document root <none> and password file <none>
I20260806 06:46:25.986933 11905 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:25.986982 11905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:25.987146 11905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:25.988199 11905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "791c0e2df45f4507b274dbbb6acaad04"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d5174c75e2fbef460bd582d9bf5be002"
server_key_iv: "64330bcc2642e2c543f942d23f1688b7"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.988559 11905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "791c0e2df45f4507b274dbbb6acaad04"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d5174c75e2fbef460bd582d9bf5be002"
server_key_iv: "64330bcc2642e2c543f942d23f1688b7"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.989797 11905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:25.990597 11919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:25.990752 11905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:25.990841 11905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "791c0e2df45f4507b274dbbb6acaad04"
format_stamp: "Formatted at 2026-08-06 06:46:25 on dist-test-slave-5l3k"
server_key: "d5174c75e2fbef460bd582d9bf5be002"
server_key_iv: "64330bcc2642e2c543f942d23f1688b7"
server_key_version: "encryptionkey@0"
I20260806 06:46:25.990911 11905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:26.004218 11905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:26.004499 11905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:26.004647 11905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:26.008576 11905 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41955
I20260806 06:46:26.008610 11971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41955 every 8 connection(s)
I20260806 06:46:26.008949 11905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:26.009634 11972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:26.012405 11972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04: Bootstrap starting.
I20260806 06:46:26.013023 11972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:26.013320 11972 log.cc:824] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:26.014002 11972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04: No bootstrap required, opened a new log
I20260806 06:46:26.015612 11972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } }
I20260806 06:46:26.015743 11972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:26.015784 11972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791c0e2df45f4507b274dbbb6acaad04, State: Initialized, Role: FOLLOWER
I20260806 06:46:26.015980 11972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } }
I20260806 06:46:26.016064 11972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:26.016115 11972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:26.016172 11972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:26.016693 11972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } }
I20260806 06:46:26.016803 11972 leader_election.cc:302] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 791c0e2df45f4507b274dbbb6acaad04; no voters:
I20260806 06:46:26.016980 11972 leader_election.cc:288] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:26.017023 11975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:26.017226 11972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:26.017235 11975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [term 1 LEADER]: Becoming Leader. State: Replica: 791c0e2df45f4507b274dbbb6acaad04, State: Running, Role: LEADER
I20260806 06:46:26.017385 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 11905
I20260806 06:46:26.017377 11975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } }
I20260806 06:46:26.017488 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:26.017707 6373 external_mini_cluster.cc:1428] Setting key ff3d665fc8d1c56c21ffa8f39571ca28
I20260806 06:46:26.017717 11975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 791c0e2df45f4507b274dbbb6acaad04. Latest consensus state: current_term: 1 leader_uuid: "791c0e2df45f4507b274dbbb6acaad04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } } }
I20260806 06:46:26.017794 11975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:26.018118 11976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "791c0e2df45f4507b274dbbb6acaad04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c0e2df45f4507b274dbbb6acaad04" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41955 } } }
I20260806 06:46:26.018255 11976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:26.018087 11982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:26.018999 11982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:26.021378 11982 catalog_manager.cc:1347] Generated new cluster ID: 284ca3d54b8c41298d3a6fc83a5d9e7b
I20260806 06:46:26.021433 11982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:26.023897 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:26.018365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.6.57.126:41955)
0806 06:46:26.018689 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:26.018698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:26.018739 (+ 41us) server_negotiation.cc:407] Connection header received
0806 06:46:26.018865 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:26.018872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:26.018945 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:26.019119 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:26.019579 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.020900 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.021831 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.022238 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.022377 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.023228 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:26.023232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:26.023233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:26.023413 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:26.023445 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:26.023456 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:26.023526 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:26.023650 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:46:26.023735 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":81,"threads_started":1}
I20260806 06:46:26.024909 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:46:26.050509 11982 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:26.051117 11982 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:26.055899 11982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 791c0e2df45f4507b274dbbb6acaad04: Generated new TSK 0
I20260806 06:46:26.056118 11982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:26.066959 11985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:26.063398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.6.57.126:41955)
0806 06:46:26.063460 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:26.063463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:26.063578 (+ 115us) server_negotiation.cc:407] Connection header received
0806 06:46:26.063654 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:26.063654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:26.063665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:26.063723 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:46:26.063859 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.064380 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.065513 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.065687 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.065741 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.065917 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:46:26.066743 (+ 826us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:46:26.066886 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 06:46:26.067037 12001 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:26.063340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41955 (local address 127.0.0.1:57744)
0806 06:46:26.063510 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:26.063521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:26.063587 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:26.063734 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:26.063735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:26.063737 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:46:26.063799 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:26.063801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.064455 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.064456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:26.065428 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:26.065431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.065765 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.065766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:26.065827 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.065832 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:46:26.066842 (+ 1010us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 06:46:26.066935 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.57.126:41955: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
W20260806 06:46:26.067237 11985 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57744: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42697/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260806 06:46:26.067631 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 11905
E20260806 06:46:26.078284 12002 webserver.cc:533] Webserver: sslize failed:
E20260806 06:46:26.078341 12002 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T06:46:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T06:46:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:26Z Disabled control of system clock
I20260806 06:46:26.115625 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44443
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:37863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:44443
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/testchainca.cert with env {}
W20260806 06:46:26.196731 12006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:26.196939 12006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:26.196960 12006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:26.198062 12006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:26.198096 12006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:26.198108 12006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:26.198120 12006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:26.198132 12006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 06:46:26.198144 12006 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 06:46:26.198155 12006 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 06:46:26.199625 12006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:44443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:26.199815 12006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:26.199965 12006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:26.204572 12015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:26.204538 12012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:26.204581 12013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:26.204797 12006 server_base.cc:1034] running on GCE node
I20260806 06:46:26.205073 12006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:26.205361 12006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:26.206557 12006 hybrid_clock.cc:648] HybridClock initialized: now 1785998786206540 us; error 42 us; skew 500 ppm
I20260806 06:46:26.207784 12006 webserver.cc:460] Webserver started at http://127.6.57.126:40589/ using document root <none> and password file <none>
I20260806 06:46:26.208014 12006 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:26.208076 12006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:26.208235 12006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:26.209231 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "0c74b5b180f24b43b9007260b45fcc29"
format_stamp: "Formatted at 2026-08-06 06:46:26 on dist-test-slave-5l3k"
server_key: "fb288921d8bfefefd5f9df62a0d7683d"
server_key_iv: "ce78ee12665daca9b0fa3d87c656d196"
server_key_version: "encryptionkey@0"
I20260806 06:46:26.209601 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "0c74b5b180f24b43b9007260b45fcc29"
format_stamp: "Formatted at 2026-08-06 06:46:26 on dist-test-slave-5l3k"
server_key: "fb288921d8bfefefd5f9df62a0d7683d"
server_key_iv: "ce78ee12665daca9b0fa3d87c656d196"
server_key_version: "encryptionkey@0"
I20260806 06:46:26.211042 12006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:26.211758 12021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:26.211921 12006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:26.212015 12006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "0c74b5b180f24b43b9007260b45fcc29"
format_stamp: "Formatted at 2026-08-06 06:46:26 on dist-test-slave-5l3k"
server_key: "fb288921d8bfefefd5f9df62a0d7683d"
server_key_iv: "ce78ee12665daca9b0fa3d87c656d196"
server_key_version: "encryptionkey@0"
I20260806 06:46:26.212088 12006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:26.235466 12006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:26.235806 12006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:26.235960 12006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:26.240044 12006 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:44443
I20260806 06:46:26.240121 12074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:44443 every 8 connection(s)
I20260806 06:46:26.240451 12006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:26.241393 12075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:26.243818 12075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29: Bootstrap starting.
I20260806 06:46:26.244437 12075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:26.244735 12075 log.cc:824] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:26.245404 12075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29: No bootstrap required, opened a new log
I20260806 06:46:26.247082 12075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } }
I20260806 06:46:26.247229 12075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:26.247280 12075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c74b5b180f24b43b9007260b45fcc29, State: Initialized, Role: FOLLOWER
I20260806 06:46:26.247453 12075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } }
I20260806 06:46:26.247535 12075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:26.247619 12075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:26.247666 12075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:26.248175 12075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } }
I20260806 06:46:26.248286 12075 leader_election.cc:302] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c74b5b180f24b43b9007260b45fcc29; no voters:
I20260806 06:46:26.248456 12075 leader_election.cc:288] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:26.248479 12078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:26.248631 12078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [term 1 LEADER]: Becoming Leader. State: Replica: 0c74b5b180f24b43b9007260b45fcc29, State: Running, Role: LEADER
I20260806 06:46:26.248718 12075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:26.248790 12078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } }
I20260806 06:46:26.249140 12079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c74b5b180f24b43b9007260b45fcc29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } } }
I20260806 06:46:26.249238 12081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c74b5b180f24b43b9007260b45fcc29. Latest consensus state: current_term: 1 leader_uuid: "0c74b5b180f24b43b9007260b45fcc29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c74b5b180f24b43b9007260b45fcc29" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44443 } } }
I20260806 06:46:26.249326 12081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:26.249269 12079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:26.249760 12088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:26.250286 12088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:26.250710 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12006
I20260806 06:46:26.250813 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:26.250996 6373 external_mini_cluster.cc:1428] Setting key d102a30bf295c5c5ffd3f5488afd4217
I20260806 06:46:26.252127 12088 catalog_manager.cc:1347] Generated new cluster ID: 489048b4cb5047c1bd4d5afade88e506
I20260806 06:46:26.252179 12088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:26.259858 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:26.251484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.6.57.126:44443)
0806 06:46:26.254646 (+ 3162us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:26.254655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:26.254695 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:46:26.254808 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:26.254815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:26.254894 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:26.255033 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:26.255293 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.256648 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.257632 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.258287 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.258406 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.259157 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:26.259161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:26.259162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:26.259362 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:26.259413 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:26.259427 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:26.259496 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:26.259575 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 06:46:26.259650 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3075,"thread_start_us":106,"threads_started":1}
I20260806 06:46:26.260110 12088 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:26.260812 12088 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:26.260998 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:46:26.264324 12088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c74b5b180f24b43b9007260b45fcc29: Generated new TSK 0
I20260806 06:46:26.264485 12088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:26.337643 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:26.270810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.6.57.126:44443)
0806 06:46:26.270847 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:26.270849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:26.270952 (+ 103us) server_negotiation.cc:407] Connection header received
0806 06:46:26.271033 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:26.271034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:26.271044 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:26.271098 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0806 06:46:26.271249 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.271716 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.272894 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.273114 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.273185 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.273363 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 06:46:26.337416 (+ 64053us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:46:26.337582 (+ 166us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57724: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:46:26.337739 12098 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57724: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 06:46:26.337847 12105 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:26.270751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44443 (local address 127.0.0.1:57724)
0806 06:46:26.270890 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:26.270899 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:26.270955 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:26.271108 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:26.271109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:26.271117 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 06:46:26.271171 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:26.271174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.271801 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.271802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:26.272775 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:26.272778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:26.273203 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:26.273204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:26.273266 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:26.273271 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 06:46:26.337559 (+ 64288us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 06:46:26.337705 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.57.126:44443: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260806 06:46:26.338294 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12006
2026-08-06T06:46:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (250 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:26 dist-test-slave-5l3k krb5kdc[12117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:26 dist-test-slave-5l3k krb5kdc[12117](info): set up 2 sockets
Aug 06 06:46:26 dist-test-slave-5l3k krb5kdc[12117](info): commencing operation
krb5kdc: starting...
W20260806 06:46:27.569237 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:27 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:27.693635 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36627
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:36627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:27.776517 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:27.776726 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:27.776750 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:27.777949 12130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:27.778004 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:27.778021 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:27.778079 12130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:27.778101 12130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:27.779618 12130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:36627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:27.779904 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:27.780159 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:27.784910 12135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:27.784921 12138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:27.785063 12136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:27.785308 12130 server_base.cc:1034] running on GCE node
I20260806 06:46:27.785537 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:27.785768 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:27.786932 12130 hybrid_clock.cc:648] HybridClock initialized: now 1785998787786921 us; error 45 us; skew 500 ppm
Aug 06 06:46:27 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998787, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:27.789330 12130 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:27.789762 12130 webserver.cc:460] Webserver started at http://127.6.57.126:46261/ using document root <none> and password file <none>
I20260806 06:46:27.789986 12130 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:27.790061 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:27.790206 12130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:27.791282 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "6ae07c6de0e84c78ba8e7f265066b563"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "829715c5311fad1c09ae789d61863746"
server_key_iv: "461f4985e26821a060d98592b23b62d9"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.791636 12130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "6ae07c6de0e84c78ba8e7f265066b563"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "829715c5311fad1c09ae789d61863746"
server_key_iv: "461f4985e26821a060d98592b23b62d9"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.792912 12130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:27.793644 12145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:27.793812 12130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:27.793915 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "6ae07c6de0e84c78ba8e7f265066b563"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "829715c5311fad1c09ae789d61863746"
server_key_iv: "461f4985e26821a060d98592b23b62d9"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.793985 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:27.820196 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:27.821924 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:27.822088 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:27.825961 12130 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:36627
I20260806 06:46:27.825999 12198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:36627 every 8 connection(s)
I20260806 06:46:27.826367 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:27.827210 12199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:27.828080 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12130
I20260806 06:46:27.828215 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:27.828384 6373 external_mini_cluster.cc:1428] Setting key a8bd3fef1b358736238452b74bac1d6c
I20260806 06:46:27.830354 12199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563: Bootstrap starting.
I20260806 06:46:27.831136 12199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:27.831427 12199 log.cc:824] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:27.832073 12199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563: No bootstrap required, opened a new log
I20260806 06:46:27.834138 12199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } }
I20260806 06:46:27.834263 12199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:27.834285 12199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ae07c6de0e84c78ba8e7f265066b563, State: Initialized, Role: FOLLOWER
I20260806 06:46:27.834442 12199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } }
I20260806 06:46:27.834559 12199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:27.834594 12199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:27.834662 12199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:46:27 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:27.835325 12199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } }
I20260806 06:46:27.835436 12199 leader_election.cc:302] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ae07c6de0e84c78ba8e7f265066b563; no voters:
I20260806 06:46:27.835601 12199 leader_election.cc:288] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:27.835655 12204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:27.835839 12204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [term 1 LEADER]: Becoming Leader. State: Replica: 6ae07c6de0e84c78ba8e7f265066b563, State: Running, Role: LEADER
I20260806 06:46:27.835886 12199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:27.835944 12204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } }
I20260806 06:46:27.836297 12205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ae07c6de0e84c78ba8e7f265066b563. Latest consensus state: current_term: 1 leader_uuid: "6ae07c6de0e84c78ba8e7f265066b563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } } }
I20260806 06:46:27.836455 12206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ae07c6de0e84c78ba8e7f265066b563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ae07c6de0e84c78ba8e7f265066b563" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36627 } } }
I20260806 06:46:27.836545 12206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:27.836524 12205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:27.836771 12213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:27.837368 12213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:27.838999 12213 catalog_manager.cc:1347] Generated new cluster ID: 43cb22b7a5d14317b57b12b7113aa5ad
I20260806 06:46:27.839063 12213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:27.839751 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:27.828874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.6.57.126:36627)
0806 06:46:27.829059 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:27.829067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:27.829104 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:46:27.829587 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:27.829596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:27.829660 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:27.829814 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:27.830442 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:27.831747 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:27.832497 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:27.833094 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:27.833219 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:27.835788 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:27.835792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:27.835792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:27.835803 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:27.838126 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:27.838556 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:27.838559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:27.838560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:27.838661 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:27.838877 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:27.838878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:27.838878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:27.839183 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:27.839325 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:27.839449 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 06:46:27.839585 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260806 06:46:27.849577 12213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:27.850243 12213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:27.855571 12213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ae07c6de0e84c78ba8e7f265066b563: Generated new TSK 0
I20260806 06:46:27.855743 12213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:27.884884 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:27.974081 12225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:27.974349 12225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:27.974391 12225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:27.975773 12225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:27.975862 12225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:27.975903 12225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:27.977407 12225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:27.977679 12225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:27.977939 12225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:27.982949 12231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:27.982959 12232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:27.982954 12234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:27.983223 12225 server_base.cc:1034] running on GCE node
I20260806 06:46:27.983459 12225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:27.983708 12225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:27.984881 12225 hybrid_clock.cc:648] HybridClock initialized: now 1785998787984860 us; error 31 us; skew 500 ppm
Aug 06 06:46:27 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998787, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:27.987486 12225 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:27.987942 12225 webserver.cc:460] Webserver started at http://127.6.57.65:43263/ using document root <none> and password file <none>
I20260806 06:46:27.988205 12225 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:27.988274 12225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:27.988412 12225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:27.989428 12225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "45371794a22845a782b78e41ed6bd272"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "eca061c95c8f5cf21b2fa0eea4dd63d5"
server_key_iv: "beabafb028a823da4d71524fe2417ade"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.989776 12225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "45371794a22845a782b78e41ed6bd272"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "eca061c95c8f5cf21b2fa0eea4dd63d5"
server_key_iv: "beabafb028a823da4d71524fe2417ade"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.991271 12225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:27.992002 12241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:27.992175 12225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:27.992274 12225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "45371794a22845a782b78e41ed6bd272"
format_stamp: "Formatted at 2026-08-06 06:46:27 on dist-test-slave-5l3k"
server_key: "eca061c95c8f5cf21b2fa0eea4dd63d5"
server_key_iv: "beabafb028a823da4d71524fe2417ade"
server_key_version: "encryptionkey@0"
I20260806 06:46:27.992342 12225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:28.004356 12225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:28.006629 12225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:28.006784 12225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:28.007005 12225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:28.007309 12225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:28.007361 12225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.007401 12225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:28.007431 12225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.013232 12225 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:38703
I20260806 06:46:28.013283 12354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:38703 every 8 connection(s)
I20260806 06:46:28.013569 12225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998787, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:28.020015 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12225
I20260806 06:46:28.020143 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:28.020372 6373 external_mini_cluster.cc:1428] Setting key c68a4be376a576d831058ac48ef749ff
I20260806 06:46:28.022647 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.014216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:36071 (local address 127.6.57.126:36627)
0806 06:46:28.014296 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.014298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.014506 (+ 208us) server_negotiation.cc:407] Connection header received
0806 06:46:28.015406 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.015407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.015417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.015469 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.016040 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.016511 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.017516 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.017734 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.017805 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.019888 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.019889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.019890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.019896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.021312 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.021683 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.021684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.021685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.021745 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.021974 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.021975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.021975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.022102 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.022167 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.022438 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.022558 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:46:28.022783 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.014182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36627 (local address 127.6.57.65:36071)
0806 06:46:28.014370 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:28.014421 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:28.015317 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:28.015520 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:28.015523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:28.015797 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:28.015971 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.015975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.016584 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.016585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.017437 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.017439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.017823 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.017824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.017922 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.018430 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:28.018438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:28.019770 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:28.021405 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.021411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.021413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.021614 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.021810 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.021811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.021811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.021912 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.022170 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:28.022177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:28.022308 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:46:28.022405 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:46:28.022573 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260806 06:46:28.023213 12355 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36627
I20260806 06:46:28.023312 12355 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:28.023480 12355 heartbeater.cc:511] Master 127.6.57.126:36627 requested a full tablet report, sending...
I20260806 06:46:28.023923 12163 ts_manager.cc:194] Registered new tserver with Master: 45371794a22845a782b78e41ed6bd272 (127.6.57.65:38703)
I20260806 06:46:28.024494 12163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:36071
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:28.045549 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:28.125597 12360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:28.125818 12360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:28.125851 12360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:28.127002 12360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:28.127070 12360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:28.127112 12360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:28.128540 12360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:28.128798 12360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:28.129006 12360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:28.133687 12366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:28.133669 12369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:28.133706 12365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:28.133716 12360 server_base.cc:1034] running on GCE node
I20260806 06:46:28.134136 12360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:28.134344 12360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:28.135458 12360 hybrid_clock.cc:648] HybridClock initialized: now 1785998788135444 us; error 30 us; skew 500 ppm
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.138187 12360 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:28.138633 12360 webserver.cc:460] Webserver started at http://127.6.57.66:36471/ using document root <none> and password file <none>
I20260806 06:46:28.138830 12360 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:28.138872 12360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:28.138958 12360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:28.139892 12360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "91e600a82ace451885bf461fbad5fbb2"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "65cbca01539c5a2fc2814710d818da53"
server_key_iv: "15b5a255986002cd5f2c0039fe326a86"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.140172 12360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "91e600a82ace451885bf461fbad5fbb2"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "65cbca01539c5a2fc2814710d818da53"
server_key_iv: "15b5a255986002cd5f2c0039fe326a86"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.141330 12360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:28.141939 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.142174 12360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:28.142277 12360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "91e600a82ace451885bf461fbad5fbb2"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "65cbca01539c5a2fc2814710d818da53"
server_key_iv: "15b5a255986002cd5f2c0039fe326a86"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.142349 12360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:28.157418 12360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:28.159356 12360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:28.159507 12360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:28.159718 12360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:28.160027 12360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:28.160077 12360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.160111 12360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:28.160137 12360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.165534 12360 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:46823
I20260806 06:46:28.165668 12492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:46823 every 8 connection(s)
I20260806 06:46:28.165949 12360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:28.169991 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12360
I20260806 06:46:28.170102 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:28.170281 6373 external_mini_cluster.cc:1428] Setting key 4fe1e02b79b67005e8ab6d3af232f079
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:28.175968 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.166746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:59475 (local address 127.6.57.126:36627)
0806 06:46:28.166812 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.166813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.167022 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:46:28.167846 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.167847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.167862 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.167919 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.168442 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.168914 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.169797 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.169990 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.170095 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.172785 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.172787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.172788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.172797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.174529 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.174922 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.174924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.174924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.174988 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.175216 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.175217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.175218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.175377 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.175454 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.175734 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.175833 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:28.175992 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.166729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36627 (local address 127.6.57.66:59475)
0806 06:46:28.166875 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:28.166914 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:28.167760 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:28.167967 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:28.167970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:28.168199 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:28.168367 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.168370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.168987 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.168987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.169729 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.169731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.170104 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.170106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.170240 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.170935 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:28.170945 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:28.172657 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:28.174620 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.174625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.174627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.174852 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.175063 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.175064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.175064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.175156 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.175461 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:28.175466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:28.175575 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 06:46:28.175708 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 06:46:28.175844 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260806 06:46:28.176592 12493 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36627
I20260806 06:46:28.176712 12493 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:28.176885 12493 heartbeater.cc:511] Master 127.6.57.126:36627 requested a full tablet report, sending...
I20260806 06:46:28.177311 12163 ts_manager.cc:194] Registered new tserver with Master: 91e600a82ace451885bf461fbad5fbb2 (127.6.57.66:46823)
I20260806 06:46:28.177867 12163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:59475
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:28.195186 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:28.275607 12498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:28.275804 12498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:28.275826 12498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:28.276913 12498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:28.276952 12498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:28.276978 12498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:28.278424 12498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:28.278663 12498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:28.278887 12498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:28.283648 12503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:28.283624 12504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:28.283627 12506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:28.283874 12498 server_base.cc:1034] running on GCE node
I20260806 06:46:28.284107 12498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:28.284333 12498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:28.285498 12498 hybrid_clock.cc:648] HybridClock initialized: now 1785998788285477 us; error 38 us; skew 500 ppm
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.287922 12498 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:28.288342 12498 webserver.cc:460] Webserver started at http://127.6.57.67:43575/ using document root <none> and password file <none>
I20260806 06:46:28.288551 12498 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:28.288612 12498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:28.288748 12498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:28.289741 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "b69d9f734a72411ab538d8215268086a"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "6a21112acfcd8e35e4b329d7840ee142"
server_key_iv: "30510d220ccea444252c1f65e4518196"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.290088 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "b69d9f734a72411ab538d8215268086a"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "6a21112acfcd8e35e4b329d7840ee142"
server_key_iv: "30510d220ccea444252c1f65e4518196"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.291410 12498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:28.292076 12513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.292236 12498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.292333 12498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "b69d9f734a72411ab538d8215268086a"
format_stamp: "Formatted at 2026-08-06 06:46:28 on dist-test-slave-5l3k"
server_key: "6a21112acfcd8e35e4b329d7840ee142"
server_key_iv: "30510d220ccea444252c1f65e4518196"
server_key_version: "encryptionkey@0"
I20260806 06:46:28.292398 12498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:28.310523 12498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:28.312333 12498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:28.312486 12498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:28.312716 12498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:28.313017 12498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:28.313067 12498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.313109 12498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:28.313138 12498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:28.318698 12498 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:34341
I20260806 06:46:28.318758 12636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:34341 every 8 connection(s)
I20260806 06:46:28.319087 12498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:28.319801 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12498
I20260806 06:46:28.319928 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:28.320111 6373 external_mini_cluster.cc:1428] Setting key 400b3b00e5e7a41fce9903fdae24cb68
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:28.327708 12202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.319799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:41209 (local address 127.6.57.126:36627)
0806 06:46:28.319882 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.319885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.320120 (+ 235us) server_negotiation.cc:407] Connection header received
0806 06:46:28.321249 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.321250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.321260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.321306 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.321845 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.322311 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.323251 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.323466 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.323531 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.325465 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.325466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.325467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.325472 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.326665 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.327001 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.327002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.327002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.327061 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.327259 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.327259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.327260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.327351 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.327407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.327599 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.327650 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:46:28.327865 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.319744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36627 (local address 127.6.57.67:41209)
0806 06:46:28.320001 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:28.320041 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:28.321161 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:28.321360 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:28.321363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:28.321591 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:28.321777 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.321780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.322380 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.322380 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.323173 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:28.323176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.323535 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.323535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:28.323626 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.324085 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:28.324093 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:28.325381 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:28.326727 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.326732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.326733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.326919 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.327116 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:28.327117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:28.327117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:28.327208 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:28.327408 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:28.327412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:28.327507 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:46:28.327576 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:46:28.327674 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
I20260806 06:46:28.328329 12637 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36627
I20260806 06:46:28.328419 12637 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:28.328570 12637 heartbeater.cc:511] Master 127.6.57.126:36627 requested a full tablet report, sending...
I20260806 06:46:28.328943 12163 ts_manager.cc:194] Registered new tserver with Master: b69d9f734a72411ab538d8215268086a (127.6.57.67:34341)
I20260806 06:46:28.329324 12163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:41209
I20260806 06:46:28.331777 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/test-admin.keytab.
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.348281 6373 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 06:46:28.349895 12644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:28.349162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.6.57.65:38703)
0806 06:46:28.349347 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.349351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.349362 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:28.349386 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.349387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.349400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.349468 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.349805 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45142: BlockingRecv error: recv got EOF from 127.0.0.1:45142 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260806 06:46:28.349953 6373 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.351011 6373 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.351900 6373 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:28.359103 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.352414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.6.57.65:38703)
0806 06:46:28.352467 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.352468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.352506 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:46:28.352557 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.352558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.352568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.352615 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.352969 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.353665 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.354345 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.354706 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.354763 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.356748 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.356754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.356755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.356761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.358107 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.358417 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.358419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.358419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.358472 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.358675 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.358676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.358676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.358799 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.358901 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.358965 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.359036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 06:46:28.360790 12644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:28.360242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45172 (local address 127.6.57.65:38703)
0806 06:46:28.360277 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.360278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.360324 (+ 46us) server_negotiation.cc:407] Connection header received
0806 06:46:28.360370 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.360371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.360378 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.360419 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.360758 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45172: BlockingRecv error: recv got EOF from 127.0.0.1:45172 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:46:28.360911 6373 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc' of type FILE
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.361780 6373 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.362653 6373 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:28.369385 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.363188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45180 (local address 127.6.57.65:38703)
0806 06:46:28.363236 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.363238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.363317 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:46:28.363366 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.363367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.363374 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.363420 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.363770 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.364210 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.364924 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.365103 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.365152 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.367195 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.367196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.367197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.367203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.368305 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.368664 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.368665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.368666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.368718 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.368915 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.368916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.368916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.369004 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.369091 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.369231 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.369300 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 06:46:28.370803 12644 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:28.370295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45182 (local address 127.6.57.65:38703)
0806 06:46:28.370327 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.370329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.370369 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:46:28.370427 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.370428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.370434 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.370556 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.370760 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45182: BlockingRecv error: recv got EOF from 127.0.0.1:45182 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:46:28.371079 6373 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785998763187424-6373-0/krb5kdc/krb5cc)
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.371973 6373 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:28.372743 6373 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:46:28.379218 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:28.373183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45194 (local address 127.6.57.65:38703)
0806 06:46:28.373214 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:28.373216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:28.373285 (+ 69us) server_negotiation.cc:407] Connection header received
0806 06:46:28.373352 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:28.373353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:28.373361 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:28.373408 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:28.373748 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.374160 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.374900 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:28.375080 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:28.375165 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:28.376951 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:28.376952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:28.376953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:28.376959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:28.378055 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.378411 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.378412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.378412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.378464 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:28.378709 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:28.378710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:28.378710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:28.378802 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:28.378905 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:28.379043 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:46:28.379134 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:46:28.379791 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12225
I20260806 06:46:28.385607 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12360
I20260806 06:46:28.390604 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12498
I20260806 06:46:28.395336 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12130
2026-08-06T06:46:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1880 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12658](info): set up 2 sockets
Aug 06 06:46:28 dist-test-slave-5l3k krb5kdc[12658](info): commencing operation
krb5kdc: starting...
W20260806 06:46:29.549335 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 06:46:29.713270 6373 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.164s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 06:46:29.940603 6373 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.227s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 06:46:30.165725 6373 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.225s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/test-user.keytab.
W20260806 06:46:30.877641 6373 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.712s user 0.000s sys 0.002s
Aug 06 06:46:30 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab.
Entry for principal oryx/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab.
I20260806 06:46:30.923794 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43343
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab
--principal=oryx/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.004961 12673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.005183 12673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.005219 12673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.006341 12673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:31.006404 12673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.006429 12673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.006450 12673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:31.006497 12673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:31.007954 12673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab
--principal=oryx/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:31.008224 12673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.008447 12673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.013199 12682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.013201 12679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.013288 12680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.013780 12673 server_base.cc:1034] running on GCE node
I20260806 06:46:31.013975 12673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.014204 12673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.015338 12673 hybrid_clock.cc:648] HybridClock initialized: now 1785998791015329 us; error 32 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.023011 12673 init.cc:377] Logged in from keytab as oryx/127.6.57.126@KRBTEST.COM (short username oryx)
I20260806 06:46:31.023526 12673 webserver.cc:460] Webserver started at http://127.6.57.126:36861/ using document root <none> and password file <none>
I20260806 06:46:31.023778 12673 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.023847 12673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.023991 12673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.025055 12673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "3032a4ae73744fd593e028910ee7ed2f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "e55b0f74eaea7bfa010e8316b12a15ae"
server_key_iv: "36faa0283be59a8651f85ebb8b90fa79"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.025410 12673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "3032a4ae73744fd593e028910ee7ed2f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "e55b0f74eaea7bfa010e8316b12a15ae"
server_key_iv: "36faa0283be59a8651f85ebb8b90fa79"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.026800 12673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:31.027542 12689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.027714 12673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:31.027817 12673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "3032a4ae73744fd593e028910ee7ed2f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "e55b0f74eaea7bfa010e8316b12a15ae"
server_key_iv: "36faa0283be59a8651f85ebb8b90fa79"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.027886 12673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.045758 12673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.047643 12673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:31.047799 12673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.051862 12673 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:43343
I20260806 06:46:31.051898 12742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:43343 every 8 connection(s)
I20260806 06:46:31.052250 12673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:31.053319 12743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:31.055743 12743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Bootstrap starting.
I20260806 06:46:31.056335 12743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:31.056628 12743 log.cc:824] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:31.057312 12743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: No bootstrap required, opened a new log
I20260806 06:46:31.058684 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12673
I20260806 06:46:31.058812 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:31.059038 6373 external_mini_cluster.cc:1428] Setting key cf71255ec0c051d02b24a93c9b003f84
I20260806 06:46:31.059464 12743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } }
I20260806 06:46:31.059674 12743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:31.059774 12743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3032a4ae73744fd593e028910ee7ed2f, State: Initialized, Role: FOLLOWER
I20260806 06:46:31.059954 12743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } }
I20260806 06:46:31.060043 12743 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:31.060086 12743 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:31.060138 12743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:31.061048 12743 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } }
I20260806 06:46:31.061172 12743 leader_election.cc:302] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3032a4ae73744fd593e028910ee7ed2f; no voters:
I20260806 06:46:31.061340 12743 leader_election.cc:288] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:31.061375 12748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:31.061491 12748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 LEADER]: Becoming Leader. State: Replica: 3032a4ae73744fd593e028910ee7ed2f, State: Running, Role: LEADER
I20260806 06:46:31.061632 12743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:31.061641 12748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } }
I20260806 06:46:31.062222 12750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3032a4ae73744fd593e028910ee7ed2f. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } }
I20260806 06:46:31.062341 12750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:31.062223 12749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } }
I20260806 06:46:31.062463 12749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:31.062608 12758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:31.063198 12758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:31.064965 12758 catalog_manager.cc:1347] Generated new cluster ID: b78000ac615f44db8da413ede9ff4642
I20260806 06:46:31.065022 12758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:31.070379 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.059586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.6.57.126:43343)
0806 06:46:31.059790 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.059800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.059848 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:46:31.059967 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.059973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.060023 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.060191 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.060882 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.062065 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.063411 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.064074 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.064191 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.066871 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.066875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.066876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.066888 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.068787 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.069169 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.069173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.069173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.069278 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.069492 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.069493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.069494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.069737 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.069945 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.070028 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.070182 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260806 06:46:31.070930 12758 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:31.071591 12758 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:31.078773 12758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Generated new TSK 0
I20260806 06:46:31.078949 12758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
I20260806 06:46:31.111583 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.216176 12770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.216372 12770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.216394 12770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.217594 12770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.217643 12770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.217669 12770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:31.219213 12770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:31.219467 12770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.219702 12770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.224587 12775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.224540 12776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.224539 12778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.225245 12770 server_base.cc:1034] running on GCE node
I20260806 06:46:31.225451 12770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.225708 12770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.226878 12770 hybrid_clock.cc:648] HybridClock initialized: now 1785998791226860 us; error 30 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.229619 12770 init.cc:377] Logged in from keytab as oryx/127.6.57.65@KRBTEST.COM (short username oryx)
I20260806 06:46:31.230290 12770 webserver.cc:460] Webserver started at http://127.6.57.65:46681/ using document root <none> and password file <none>
I20260806 06:46:31.230635 12770 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.230715 12770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.230865 12770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.232255 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "3e4edf0ebacaf038dea7d9bb9f1bcfb5"
server_key_iv: "03f8a0e52be74c7c9eeb76d0e671cde8"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.232678 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "3e4edf0ebacaf038dea7d9bb9f1bcfb5"
server_key_iv: "03f8a0e52be74c7c9eeb76d0e671cde8"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.234416 12770 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 06:46:31.235390 12785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.235595 12770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:31.235709 12770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "3e4edf0ebacaf038dea7d9bb9f1bcfb5"
server_key_iv: "03f8a0e52be74c7c9eeb76d0e671cde8"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.235791 12770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.256917 12770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.259034 12770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:31.259217 12770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.259470 12770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:31.259820 12770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:31.259891 12770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.259936 12770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:31.259964 12770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.266575 12770 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:41819
I20260806 06:46:31.266636 12908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:41819 every 8 connection(s)
I20260806 06:46:31.266957 12770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:31.267526 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12770
I20260806 06:46:31.267653 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:31.267925 6373 external_mini_cluster.cc:1428] Setting key 1464f52490e0da12f48df391b531e59f
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:46:31.277575 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.267675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:39623 (local address 127.6.57.126:43343)
0806 06:46:31.267738 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.267741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.268020 (+ 279us) server_negotiation.cc:407] Connection header received
0806 06:46:31.268712 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.268713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.268730 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.268786 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.269831 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.270634 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.271606 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.271880 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.271966 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.274507 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.274509 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.274510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.274519 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.276052 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.276509 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.276510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.276511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.276581 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.276846 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.276847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.276847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.276970 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.277051 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.277368 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.277464 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:46:31.277778 12911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.267648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:39623)
0806 06:46:31.267858 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.267914 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.268586 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.268879 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.268882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.269498 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.269713 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.269718 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.270747 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.270748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.271528 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.271530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.271974 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.271976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.272135 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.272698 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.272709 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.274373 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.276156 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.276165 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.276167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.276411 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.276653 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.276654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.276655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.276772 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.277050 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.277058 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.277202 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.277328 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.277504 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260806 06:46:31.278306 12909 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:31.278393 12909 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:31.278605 12909 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:31.279145 12706 ts_manager.cc:194] Registered new tserver with Master: 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.66@KRBTEST.COM" created.
I20260806 06:46:31.279860 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:39623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
I20260806 06:46:31.293737 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.376951 12914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.377141 12914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.377164 12914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.378298 12914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.378338 12914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.378371 12914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:31.379868 12914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:31.380061 12914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.380228 12914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.385238 12919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.385217 12920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.385191 12922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.385196 12914 server_base.cc:1034] running on GCE node
I20260806 06:46:31.385628 12914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.385905 12914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.387079 12914 hybrid_clock.cc:648] HybridClock initialized: now 1785998791387068 us; error 37 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.389676 12914 init.cc:377] Logged in from keytab as oryx/127.6.57.66@KRBTEST.COM (short username oryx)
I20260806 06:46:31.390197 12914 webserver.cc:460] Webserver started at http://127.6.57.66:41949/ using document root <none> and password file <none>
I20260806 06:46:31.390463 12914 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.390604 12914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.390780 12914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.391922 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "ed661da5b2654ac2afe7e25d3be8237b"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "4c72cdb403dd9d65b7ab3ccc6fdf4188"
server_key_iv: "e5c3ec938232870779d902ba9367647d"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.392313 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "ed661da5b2654ac2afe7e25d3be8237b"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "4c72cdb403dd9d65b7ab3ccc6fdf4188"
server_key_iv: "e5c3ec938232870779d902ba9367647d"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.393797 12914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:31.394596 12929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.394783 12914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:31.394892 12914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "ed661da5b2654ac2afe7e25d3be8237b"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "4c72cdb403dd9d65b7ab3ccc6fdf4188"
server_key_iv: "e5c3ec938232870779d902ba9367647d"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.394982 12914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.405809 12914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.407737 12914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:31.407912 12914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.408150 12914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:31.408516 12914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:31.408578 12914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.408627 12914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:31.408666 12914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.415603 12914 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:37667
I20260806 06:46:31.415669 13046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:37667 every 8 connection(s)
I20260806 06:46:31.416040 12914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:31.418591 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 12914
I20260806 06:46:31.418718 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:31.418951 6373 external_mini_cluster.cc:1428] Setting key 6658e79e29f7b74f9d8116e645f56ba2
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:31.426537 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.416769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37547 (local address 127.6.57.126:43343)
0806 06:46:31.416821 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.416824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.416999 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:46:31.417464 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.417465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.417492 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.417560 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.418471 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.419364 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.420548 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.420894 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.420986 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.423616 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.423618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.423619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.423628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.425134 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.425574 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.425575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.425576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.425634 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.425879 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.425880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.425880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.426001 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.426077 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.426335 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.426416 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:31.426707 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.416719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:37547)
0806 06:46:31.416884 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.416925 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.417374 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.417617 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.417620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.418172 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.418360 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.418364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.419495 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.419496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.420453 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.420456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.421002 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.421004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.421149 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.421739 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.421750 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.423488 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.425253 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.425260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.425262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.425483 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.425699 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.425700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.425700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.425808 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.426070 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.426076 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.426207 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.426301 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.426456 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260806 06:46:31.427295 13047 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:31.427392 13047 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:31.427554 13047 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:31.427944 12706 ts_manager.cc:194] Registered new tserver with Master: ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667)
WARNING: no policy specified for oryx/127.6.57.67@KRBTEST.COM; defaulting to no policy
I20260806 06:46:31.428426 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:37547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
I20260806 06:46:31.445293 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.526064 13052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.526263 13052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.526285 13052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.527452 13052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.527494 13052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.527520 13052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:31.528904 13052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:31.529134 13052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.529341 13052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.534302 13058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.534313 13057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.534446 13052 server_base.cc:1034] running on GCE node
W20260806 06:46:31.534365 13060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.534777 13052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.535020 13052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.536188 13052 hybrid_clock.cc:648] HybridClock initialized: now 1785998791536180 us; error 43 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.538612 13052 init.cc:377] Logged in from keytab as oryx/127.6.57.67@KRBTEST.COM (short username oryx)
I20260806 06:46:31.539094 13052 webserver.cc:460] Webserver started at http://127.6.57.67:36711/ using document root <none> and password file <none>
I20260806 06:46:31.539327 13052 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.539402 13052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.539543 13052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.540514 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "25bda12ce5b54fc28ddb550965e56938"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "6bb8d4dc2dc156c08aa71c06606b2667"
server_key_iv: "8f395734bacd91706db9ec334b0d865e"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.540866 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "25bda12ce5b54fc28ddb550965e56938"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "6bb8d4dc2dc156c08aa71c06606b2667"
server_key_iv: "8f395734bacd91706db9ec334b0d865e"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.542145 13052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:31.542899 13067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.543054 13052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.543148 13052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "25bda12ce5b54fc28ddb550965e56938"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "6bb8d4dc2dc156c08aa71c06606b2667"
server_key_iv: "8f395734bacd91706db9ec334b0d865e"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.543232 13052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.566586 13052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.568334 13052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:31.568488 13052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.568712 13052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:31.569016 13052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:31.569068 13052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.569110 13052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:31.569139 13052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.574923 13052 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:46355
I20260806 06:46:31.574962 13183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:46355 every 8 connection(s)
I20260806 06:46:31.575304 13052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:31.580164 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13052
I20260806 06:46:31.580281 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:31.580457 6373 external_mini_cluster.cc:1428] Setting key 4192fef607eb7ceaa08d362c4a410c4d
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:31.584200 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.576008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:54359 (local address 127.6.57.126:43343)
0806 06:46:31.576052 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.576055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.576247 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:46:31.576696 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.576697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.576708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.576759 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.577629 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.578113 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.579227 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.579413 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.579469 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.581908 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.581909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.581910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.581916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.583101 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.583472 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.583473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.583473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.583521 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.583721 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.583721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.583721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.583816 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.583866 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.584080 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.584138 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:31.584450 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.575980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:54359)
0806 06:46:31.576132 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.576166 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.576616 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.576809 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.576812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.577361 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.577545 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.577549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.578186 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.578187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.579158 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.579161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.579485 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.579486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.579586 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.580089 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.580100 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.581830 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.583175 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.583181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.583182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.583408 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.583574 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.583575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.583575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.583670 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.583867 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.583871 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.583976 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.584055 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.584174 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260806 06:46:31.584967 13184 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:31.585072 13184 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:31.585245 13184 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:31.585615 12706 ts_manager.cc:194] Registered new tserver with Master: 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:31.585979 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:54359
I20260806 06:46:31.592085 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.57.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.125.keytab.
Entry for principal oryx/127.6.57.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.125.keytab.
I20260806 06:46:31.616878 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:42589
--webserver_interface=127.6.57.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.125.keytab
--principal=oryx/127.6.57.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43343,127.6.57.125:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.699208 13189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.699426 13189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.699463 13189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.700584 13189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:31.700644 13189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.700670 13189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.700698 13189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:31.700721 13189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:31.702152 13189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43343,127.6.57.125:42589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.125.keytab
--principal=oryx/127.6.57.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:42589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.57.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
W20260806 06:46:31.702286 13189 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.57.126:43343,127.6.57.125:42589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 06:46:31.702432 13189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.702669 13189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.707294 13195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.707337 13194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.707351 13197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.707621 13189 server_base.cc:1034] running on GCE node
I20260806 06:46:31.707855 13189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.708153 13189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.709313 13189 hybrid_clock.cc:648] HybridClock initialized: now 1785998791709303 us; error 40 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.711840 13189 init.cc:377] Logged in from keytab as oryx/127.6.57.125@KRBTEST.COM (short username oryx)
I20260806 06:46:31.712280 13189 webserver.cc:460] Webserver started at http://127.6.57.125:39137/ using document root <none> and password file <none>
I20260806 06:46:31.712451 13189 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.712489 13189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.712572 13189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.713510 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/instance:
uuid: "a9f35b8df4714fd1875b666e57026e8a"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "bfac39ae51e070111f614d0ab83c5b45"
server_key_iv: "31fd73a7f39df8dfd05885967e785816"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.713798 13189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal/instance:
uuid: "a9f35b8df4714fd1875b666e57026e8a"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "bfac39ae51e070111f614d0ab83c5b45"
server_key_iv: "31fd73a7f39df8dfd05885967e785816"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.715121 13189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:31.715796 13205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.715989 13189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:31.716058 13189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "a9f35b8df4714fd1875b666e57026e8a"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "bfac39ae51e070111f614d0ab83c5b45"
server_key_iv: "31fd73a7f39df8dfd05885967e785816"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.716107 13189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.726312 13189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.728183 13189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:31.728289 13189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.733027 13189 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.125:42589
I20260806 06:46:31.733131 13258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.125:42589 every 8 connection(s)
I20260806 06:46:31.733420 13189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
I20260806 06:46:31.734405 13259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:31.735983 13259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:46:31.741518 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13189
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.125@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:31.745143 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.736657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37010 (local address 127.6.57.126:43343)
0806 06:46:31.736725 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.736727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.736903 (+ 176us) server_negotiation.cc:407] Connection header received
0806 06:46:31.737355 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.737356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.737367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.737436 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.738007 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.738601 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.739689 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.739897 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.739978 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.742318 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.742319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.742320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.742327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.743748 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.744173 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.744175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.744175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.744235 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.744467 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.744468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.744468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.744629 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.744707 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.744964 (+ 257us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.745047 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:46:31.745343 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.736611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37010)
0806 06:46:31.736777 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.736819 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.737267 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.737487 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.737490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.737746 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.737929 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.737933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.738681 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.738682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.739594 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.739597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.740007 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.740008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.740117 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.740655 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.740663 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.742224 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.743855 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.743862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.743864 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.744089 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.744297 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.744298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.744298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.744401 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.744698 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.744704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.744836 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.744941 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.745104 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260806 06:46:31.746028 13259 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.125@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
I20260806 06:46:31.750954 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.742084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.6.57.125:42589)
0806 06:46:31.742220 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.742225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.742592 (+ 367us) server_negotiation.cc:407] Connection header received
0806 06:46:31.742615 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.742617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.742629 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.742713 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.743295 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.743952 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.744670 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.744979 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.745051 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.747783 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.747790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.747790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.747798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.749676 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.750013 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.750017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.750017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.750114 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.750318 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.750319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.750319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.750510 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.750644 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.750744 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.750845 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260806 06:46:31.753012 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.746290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35102)
0806 06:46:31.746363 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.746376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.746405 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.746649 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.746651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.746794 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.746855 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.746856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.747779 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.747780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.748529 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.748532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.748957 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.748958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.749031 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.749324 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.749330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.750673 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.752347 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.752347 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.752348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.752497 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.752633 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.752634 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.752634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.752673 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.752813 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.752814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.752843 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.752898 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.752958 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 06:46:31.753180 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.746355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.6.57.125:42589)
0806 06:46:31.746531 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.746532 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.746538 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:46:31.746553 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.746553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.746561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.746632 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.746936 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.747685 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.748604 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.748888 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.748941 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.750750 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.750751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.750751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.750758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.752282 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.752553 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.752554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.752554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.752598 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.752705 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.752705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.752705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.752781 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.752821 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.752910 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.753000 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260806 06:46:31.754750 13259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Bootstrap starting.
I20260806 06:46:31.755414 13259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:31.755652 13259 log.cc:824] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:31.756335 13259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: No bootstrap required, opened a new log
I20260806 06:46:31.757676 13259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } }
I20260806 06:46:31.757808 13259 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:31.757833 13259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9f35b8df4714fd1875b666e57026e8a, State: Initialized, Role: FOLLOWER
I20260806 06:46:31.757915 13259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } }
I20260806 06:46:31.758152 13267 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } } }
I20260806 06:46:31.758210 13259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:31.758255 13267 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:46:31.759835 13278 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:31.759908 13278 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:31.761387 13189 master_runner.cc:428] Detected that this master a9f35b8df4714fd1875b666e57026e8a is joining an existing cluster
I20260806 06:46:31.761438 13189 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.57.125:42589
I20260806 06:46:31.769397 13285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.762461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35106)
0806 06:46:31.762703 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.762713 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.762784 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.762939 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.762939 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.763199 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.763259 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.763261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.764148 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.764148 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.764957 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.764959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.765348 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.765349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.765422 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.765779 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.765783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.766145 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.768204 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.768205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.768384 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.768618 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768618 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.768619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.768669 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.768982 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.768983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.769057 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.769131 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.769241 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":2,"spinlock_wait_cycles":11008,"thread_start_us":198,"threads_started":1}
I20260806 06:46:31.769799 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.762574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37018 (local address 127.6.57.126:43343)
0806 06:46:31.762627 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.762628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.762759 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:46:31.762835 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.762836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.762847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.762912 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.763294 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.763846 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.764577 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.764777 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.764844 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.766140 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.766141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.766142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.766149 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.768494 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768869 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.768870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.768871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.768947 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.769170 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.769171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.769172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.769300 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.769345 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.769600 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.769700 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:31.769474 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.762584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35106 (local address 127.6.57.125:42589)
0806 06:46:31.762623 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.762626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.762763 (+ 137us) server_negotiation.cc:407] Connection header received
0806 06:46:31.762852 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.762853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.762861 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.762887 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.763346 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.764047 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.765040 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.765277 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.765346 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.766200 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.766200 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.766201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.766207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.768104 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768466 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.768467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.768468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.768537 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768742 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.768744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.768744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.768894 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.768970 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.769174 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.769314 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":19}
I20260806 06:46:31.769943 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.762461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37018)
0806 06:46:31.762681 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.762693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.762771 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.762916 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.762916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.763145 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.763207 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.763209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.763922 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.763923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.764483 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.764484 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.764859 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.764860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.764927 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.765557 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.765560 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.766036 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.768588 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.768589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.768589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.768800 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.769018 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.769019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.769020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.769092 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.769394 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.769396 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.769437 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.769553 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.769648 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":111,"threads_started":1}
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.126@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
I20260806 06:46:31.779580 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.772161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35108)
0806 06:46:31.772338 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.772372 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.772433 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.772604 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.772609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.772845 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.772889 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.772890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.773773 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.773774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.774332 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.774333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.774863 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.774864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.774936 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.775426 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.775431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.776949 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.778584 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.778591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.778593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.778799 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.778990 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.778991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.778991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.779050 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.779289 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.779295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.779335 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.779399 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.779487 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260806 06:46:31.779606 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.772265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.6.57.125:42589)
0806 06:46:31.772303 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.772305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.772450 (+ 145us) server_negotiation.cc:407] Connection header received
0806 06:46:31.772519 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.772520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.772531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.772584 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.772958 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.773690 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.774406 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.774751 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.774860 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.777030 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.777031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.777031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.777038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.778513 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.778868 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.778869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.778869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.778926 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.779108 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.779109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.779109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.779228 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.779292 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.779438 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.779521 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:31.780210 12706 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.57.125:42589: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a"
member_type: NON_VOTER
last_known_addr {
host: "127.6.57.125"
port: 42589
}
attrs {
promote: true
}
}
dest_uuid: "3032a4ae73744fd593e028910ee7ed2f"
cas_config_opid_index: -1
I20260806 06:46:31.780584 12706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } }
I20260806 06:46:31.781042 13287 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a9f35b8df4714fd1875b666e57026e8a (127.6.57.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } }
I20260806 06:46:31.781315 13232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:31.782058 13287 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.57.125:42589
I20260806 06:46:31.782215 13232 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 06:46:31.782320 13189 master.cc:537] Master@127.6.57.125:42589 shutting down...
I20260806 06:46:31.782277 13288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } }
I20260806 06:46:31.782356 13288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:31.782855 13287 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:46:31.783675 12693 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 06:46:31.784080 12693 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f -> Peer a9f35b8df4714fd1875b666e57026e8a (127.6.57.125:42589): Couldn't send request to peer a9f35b8df4714fd1875b666e57026e8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:46:31.785259 13189 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:31.785357 13189 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:31.785410 13189 tablet_replica.cc:330] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: stopping tablet replica
I20260806 06:46:31.787955 13189 master.cc:552] Master@127.6.57.125:42589 shutdown complete.
I20260806 06:46:31.788002 13189 master_runner.cc:305] Clearing existing system tablet
I20260806 06:46:31.788133 13189 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.788180 13189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.788858 13189 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:31.789340 13292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.789470 13189 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.789533 13189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.789551 13189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "a9f35b8df4714fd1875b666e57026e8a"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "bfac39ae51e070111f614d0ab83c5b45"
server_key_iv: "31fd73a7f39df8dfd05885967e785816"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.790169 13189 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:31.791596 13189 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:46:31.791674 13189 log.cc:1196] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 06:46:31.791864 13189 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Deleting consensus metadata
I20260806 06:46:31.791989 13189 master_runner.cc:315] Copying system tablet from 127.6.57.126:43343
I20260806 06:46:31.792483 13189 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: tablet copy: Beginning tablet copy session from remote peer at address 127.6.57.126:43343
I20260806 06:46:31.798218 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.792856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.6.57.126:43343)
0806 06:46:31.792933 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.792935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.792948 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:46:31.793025 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.793026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.793036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.793101 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.793393 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.793867 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.794597 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.794794 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.794863 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.795745 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.795746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.795747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.795753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.797102 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.797442 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.797443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.797444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.797502 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.797707 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.797708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.797708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.797825 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.797896 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.798051 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.798128 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:31.798235 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.792730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37022)
0806 06:46:31.792866 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:31.792890 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:31.792964 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:31.793104 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:31.793105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:31.793268 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:31.793311 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.793313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.793943 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.793945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.794528 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:31.794531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.794874 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.794875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:31.794939 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.795359 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:31.795364 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:31.795643 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:31.797200 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.797202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.797202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.797372 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.797573 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:31.797574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:31.797574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:31.797656 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:31.797899 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:31.797901 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:31.797943 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:46:31.798032 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:46:31.798129 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260806 06:46:31.798795 12726 tablet_copy_service.cc:140] P 3032a4ae73744fd593e028910ee7ed2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a9f35b8df4714fd1875b666e57026e8a ({username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:37022)
I20260806 06:46:31.798880 12726 tablet_copy_service.cc:161] P 3032a4ae73744fd593e028910ee7ed2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a9f35b8df4714fd1875b666e57026e8a at {username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:37022: session id = a9f35b8df4714fd1875b666e57026e8a-00000000000000000000000000000000
I20260806 06:46:31.799440 12726 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:46:31.800015 13189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:46:31.800921 13189 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: tablet copy: Starting download of 0 data blocks...
I20260806 06:46:31.801076 13189 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: tablet copy: Starting download of 1 WAL segments...
I20260806 06:46:31.802067 13189 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:46:31.802995 12726 tablet_copy_service.cc:342] P 3032a4ae73744fd593e028910ee7ed2f: Request end of tablet copy session a9f35b8df4714fd1875b666e57026e8a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:37022
I20260806 06:46:31.803077 12726 tablet_copy_service.cc:433] P 3032a4ae73744fd593e028910ee7ed2f: ending tablet copy session a9f35b8df4714fd1875b666e57026e8a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a9f35b8df4714fd1875b666e57026e8a
I20260806 06:46:31.805711 13189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.806777 13300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.806807 13302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.806885 13299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.807021 13189 server_base.cc:1034] running on GCE node
I20260806 06:46:31.807102 13189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.807225 13189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.808341 13189 hybrid_clock.cc:648] HybridClock initialized: now 1785998791808333 us; error 21 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.809737 13189 init.cc:377] Logged in from keytab as oryx/127.6.57.125@KRBTEST.COM (short username oryx)
I20260806 06:46:31.809968 13189 webserver.cc:460] Webserver started at http://127.6.57.125:36445/ using document root <none> and password file <none>
I20260806 06:46:31.810108 13189 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.810145 13189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.810858 13189 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:31.811421 13309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:31.811558 13189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:31.811609 13189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "a9f35b8df4714fd1875b666e57026e8a"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "bfac39ae51e070111f614d0ab83c5b45"
server_key_iv: "31fd73a7f39df8dfd05885967e785816"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.811661 13189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:31.821336 13189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:31.823028 13189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:31.826027 13189 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.125:42589
I20260806 06:46:31.826253 13362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.125:42589 every 8 connection(s)
I20260806 06:46:31.826574 13189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
I20260806 06:46:31.827404 13363 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:46:31.827889 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Bootstrap starting.
I20260806 06:46:31.830691 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:46:31.830947 13363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Bootstrap complete.
I20260806 06:46:31.831130 13363 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } }
I20260806 06:46:31.831209 13363 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a9f35b8df4714fd1875b666e57026e8a, State: Initialized, Role: LEARNER
I20260806 06:46:31.831243 13363 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } }
I20260806 06:46:31.831398 13363 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:31.831418 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } }
I20260806 06:46:31.831540 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: LEARNER
I20260806 06:46:31.833427 13378 catalog_manager.cc:1259] Loaded cluster ID: b78000ac615f44db8da413ede9ff4642
I20260806 06:46:31.833488 13378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: loading cluster ID for follower catalog manager: success
I20260806 06:46:31.834239 13378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: acquiring CA information for follower catalog manager: success
I20260806 06:46:31.834576 13378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:46:31.859745 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:31.852782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.6.57.125:42589)
0806 06:46:31.852896 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:31.852898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:31.852907 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:31.852930 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:31.852931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:31.852941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:31.853008 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:31.853711 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.854421 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.855515 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:31.855703 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:31.855752 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:31.857189 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:31.857191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:31.857191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:31.857197 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:31.858648 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.859044 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.859045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.859045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.859098 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:31.859301 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:31.859301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:31.859302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:31.859395 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:31.859462 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:31.859610 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 06:46:31.859669 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for oryx/127.6.57.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.124.keytab.
Entry for principal oryx/127.6.57.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.124.keytab.
I20260806 06:46:31.889228 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:42415
--webserver_interface=127.6.57.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.124.keytab
--principal=oryx/127.6.57.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:31.984035 13382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:31.984238 13382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:31.984259 13382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:31.985337 13382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:31.985368 13382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:31.985383 13382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:31.985394 13382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:31.985406 13382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:31.986879 13382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.124.keytab
--principal=oryx/127.6.57.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:42415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.57.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:31.987068 13382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:31.987234 13382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:31.991791 13390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.991896 13388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:31.991809 13387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:31.992093 13382 server_base.cc:1034] running on GCE node
I20260806 06:46:31.992338 13382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:31.992636 13382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:31.993786 13382 hybrid_clock.cc:648] HybridClock initialized: now 1785998791993766 us; error 34 us; skew 500 ppm
Aug 06 06:46:31 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:31.996456 13382 init.cc:377] Logged in from keytab as oryx/127.6.57.124@KRBTEST.COM (short username oryx)
I20260806 06:46:31.996948 13382 webserver.cc:460] Webserver started at http://127.6.57.124:45051/ using document root <none> and password file <none>
I20260806 06:46:31.997171 13382 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:31.997221 13382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:31.997385 13382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:31.998623 13382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/instance:
uuid: "b25ebdad004a49358fe7c1fe808f574f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "ae61f344ab5caecb41296bcc8c95af45"
server_key_iv: "108b346bb270bc40ee8d39d68fd2c9b5"
server_key_version: "encryptionkey@0"
I20260806 06:46:31.999013 13382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal/instance:
uuid: "b25ebdad004a49358fe7c1fe808f574f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "ae61f344ab5caecb41296bcc8c95af45"
server_key_iv: "108b346bb270bc40ee8d39d68fd2c9b5"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.000460 13382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:32.001184 13397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.001355 13382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:32.001456 13382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "b25ebdad004a49358fe7c1fe808f574f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "ae61f344ab5caecb41296bcc8c95af45"
server_key_iv: "108b346bb270bc40ee8d39d68fd2c9b5"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.001549 13382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.012558 13382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:32.014904 13382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:32.015062 13382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:32.019237 13382 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.124:42415
I20260806 06:46:32.019271 13450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.124:42415 every 8 connection(s)
I20260806 06:46:32.019636 13382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
I20260806 06:46:32.020519 13451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:32.022207 13451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:46:32.024617 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13382
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.124@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
I20260806 06:46:32.032634 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.023072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.6.57.126:43343)
0806 06:46:32.023124 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.023126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.023136 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:32.023597 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.023598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.023608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.023662 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.024208 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.024742 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.026317 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.026588 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.026681 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.029093 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.029095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.029095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.029103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.030792 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.031498 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.031499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.031499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.031557 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.031848 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.031849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.031850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.032001 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.032064 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.032486 (+ 422us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.032558 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:32.032878 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.022857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37034)
0806 06:46:32.023026 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.023060 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.023510 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.023713 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.023716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.023944 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.024134 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.024138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.024861 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.024862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.026234 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.026238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.026691 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.026692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.026818 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.027261 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.027271 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.028995 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.030887 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.030894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.030897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.031418 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.031629 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.031629 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.031630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.031776 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.032087 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.032095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.032262 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.032442 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.032623 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260806 06:46:32.033557 13451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:46:32.034355 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.025128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.6.57.124:42415)
0806 06:46:32.025264 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.025273 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.025287 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:46:32.025306 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.025308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.025323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.025387 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.026134 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.026888 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.027863 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.028248 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.028337 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.030915 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.030920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.030921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.030928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.032866 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.033253 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.033256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.033256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.033321 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.033721 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.033721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.033722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.033900 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.034067 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.034127 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.034229 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.124@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
I20260806 06:46:32.039798 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.033823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35136)
0806 06:46:32.033868 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.033878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.033913 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.034041 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.034042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.034220 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.034270 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.034272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.035065 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.035065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.035535 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.035536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.035843 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.035843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.035895 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.036285 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.036289 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.037425 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.038936 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.038937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.038937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.039200 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.039359 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.039360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.039360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.039410 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.039583 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.039583 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.039624 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.039672 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.039738 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 06:46:32.039842 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.033883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35136 (local address 127.6.57.125:42589)
0806 06:46:32.033923 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.033925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.033933 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:32.033965 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.033966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.033978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.034041 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.034342 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.034997 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.035597 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.035785 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.035832 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.037497 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.037498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.037498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.037503 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.038859 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.039252 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.039253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.039253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.039304 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.039448 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.039448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.039448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.039538 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.039583 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.039696 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.039764 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:32.040225 13451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.124@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
I20260806 06:46:32.045717 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.040511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.6.57.124:42415)
0806 06:46:32.040561 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.040563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.040570 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:32.040607 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.040608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.040618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.040668 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.040844 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.041294 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.041845 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.042020 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.042066 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.043634 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.043636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.043636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.043642 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.044766 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.045071 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.045071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.045071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.045117 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.045256 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.045256 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.045257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.045367 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.045435 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.045594 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.045664 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:46:32.045830 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.040462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39260)
0806 06:46:32.040510 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.040521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.040558 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.040673 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.040673 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.040778 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.040805 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.040806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.041359 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.041359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.041785 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.041786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.042075 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.042075 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.042109 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.042390 (+ 281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.042394 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.043552 (+ 1158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.044834 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.044835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.044835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.044992 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.045168 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.045169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.045169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.045220 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.045429 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.045429 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.045492 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.045561 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.045726 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":48}
I20260806 06:46:32.047226 13451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Bootstrap starting.
I20260806 06:46:32.048045 13451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:32.048269 13451 log.cc:824] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:32.048938 13451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: No bootstrap required, opened a new log
I20260806 06:46:32.050395 13451 raft_consensus.cc:348] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } }
I20260806 06:46:32.050566 13451 raft_consensus.cc:374] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:32.050593 13451 raft_consensus.cc:729] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b25ebdad004a49358fe7c1fe808f574f, State: Initialized, Role: FOLLOWER
I20260806 06:46:32.050710 13451 consensus_queue.cc:260] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } }
I20260806 06:46:32.050937 13456 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } } }
I20260806 06:46:32.050990 13451 sys_catalog.cc:564] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:32.051054 13456 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:46:32.052740 13469 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:46:32.052796 13469 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:46:32.056270 13382 master_runner.cc:428] Detected that this master b25ebdad004a49358fe7c1fe808f574f is joining an existing cluster
I20260806 06:46:32.056313 13382 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.57.124:42415
I20260806 06:46:32.064186 13476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37038)
0806 06:46:32.057411 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.057422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.057467 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.057669 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.057670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.057940 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.058000 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.058001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.059033 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.059034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.059994 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.059996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060514 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.060564 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.061049 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.061054 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.061368 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.063190 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.063214 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.063215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.063412 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.063601 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.063602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.063602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.063647 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.063866 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.063866 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.063918 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.064003 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.064115 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":29,"thread_start_us":89,"threads_started":1}
I20260806 06:46:32.064196 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37038 (local address 127.6.57.126:43343)
0806 06:46:32.057463 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.057465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.057473 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:32.057555 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.057556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.057568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.057601 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.058128 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.058926 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060097 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060392 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060430 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.061463 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.061464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.061465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.061472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.063113 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.063467 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.063468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.063468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.063535 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.063686 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.063687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.063687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.063808 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.063878 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.064029 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.064108 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998791, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.126@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
I20260806 06:46:32.085589 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12770
I20260806 06:46:32.086831 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.066215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39286)
0806 06:46:32.066391 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.066401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.066454 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.066656 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.066657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.066813 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.066852 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.066853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.067536 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.067537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.068359 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.068361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.068674 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.068675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.068739 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.069073 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.069077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.070336 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.085605 (+ 15269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.085607 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.085608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.085851 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.086093 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.086095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.086163 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.086464 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.086465 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.086544 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.086597 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.086725 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260806 06:46:32.087272 13477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39276)
0806 06:46:32.057716 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.057737 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.057817 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.057959 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.057960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.058218 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.058279 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.058281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.059115 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.059115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.059994 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.059996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060464 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.060534 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.083326 (+ 22792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.083332 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.083821 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.086190 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.086191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.086379 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.086611 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086611 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.086611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.086660 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.086911 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.086912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.086964 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.087024 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.087135 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260806 06:46:32.087301 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.066266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.6.57.124:42415)
0806 06:46:32.066405 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.066407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.066438 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:46:32.066569 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.066573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.066583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.066637 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.066974 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.067466 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.068430 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.068610 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.068656 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.070419 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.070420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.070420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.070427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.085496 (+ 15069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.085944 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.085945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.085945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.086001 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086238 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.086239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.086240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.086380 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.087040 (+ 660us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.087073 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.087188 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260806 06:46:32.087553 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.6.57.124:42415)
0806 06:46:32.057547 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.057549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.057799 (+ 250us) server_negotiation.cc:407] Connection header received
0806 06:46:32.057877 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.057877 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.057885 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.057911 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.058356 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.059035 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060086 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060357 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060394 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.083919 (+ 23525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.083920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.083921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.083926 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.086116 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086450 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.086451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.086451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.086542 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.086720 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.086720 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.086721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.086822 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.086877 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.087442 (+ 565us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.087496 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:32.089033 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.6.57.125:42589)
0806 06:46:32.057330 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.057332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.057582 (+ 250us) server_negotiation.cc:407] Connection header received
0806 06:46:32.057692 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.057693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.057704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.057744 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.058246 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.059048 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060168 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060465 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060509 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.083612 (+ 23103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.083614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.083615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.083624 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.085370 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.088172 (+ 2802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.088173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.088174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.088247 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.088463 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.088464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.088465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.088602 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.088683 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.088861 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.088946 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:32.089112 13475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.057279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35146)
0806 06:46:32.057520 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.057529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.057623 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.057843 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.057844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.058073 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.058150 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.058152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.059145 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.059146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.060061 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.060064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.060590 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.060591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.060666 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.083022 (+ 22356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.083028 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.083472 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.087837 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.087838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.087839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.088065 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.088327 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.088328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.088329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.088394 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.088674 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.088675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.088750 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.088835 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.088942 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260806 06:46:32.095566 12706 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.57.124:42415: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f"
member_type: NON_VOTER
last_known_addr {
host: "127.6.57.124"
port: 42415
}
attrs {
promote: true
}
}
dest_uuid: "3032a4ae73744fd593e028910ee7ed2f"
cas_config_opid_index: 5
I20260806 06:46:32.095870 12706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } }
I20260806 06:46:32.096285 13287 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b25ebdad004a49358fe7c1fe808f574f (127.6.57.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.096891 13424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:32.097375 13287 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.57.124:42415
I20260806 06:46:32.097452 13287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.097514 13287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.098026 13424 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:46:32.098400 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:41819
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:43343
--webserver_port=46681
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:46:32.098472 13382 master.cc:537] Master@127.6.57.124:42415 shutting down...
I20260806 06:46:32.098897 13288 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:46:32.099737 12691 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f -> Peer b25ebdad004a49358fe7c1fe808f574f (127.6.57.124:42415): Couldn't send request to peer b25ebdad004a49358fe7c1fe808f574f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 06:46:32.103200 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.096418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35158)
0806 06:46:32.099208 (+ 2790us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.099220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.099262 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.099483 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.099483 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.099486 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.099531 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.099533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.100483 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.100485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.101775 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.101779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.102853 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.102854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.102938 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.102987 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.103064 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.103107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2770}
I20260806 06:46:32.103404 13382 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:32.103502 13382 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:32.103550 13382 tablet_replica.cc:330] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: stopping tablet replica
I20260806 06:46:32.104457 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.096541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.6.57.125:42589)
0806 06:46:32.099340 (+ 2799us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.099342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.099350 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:32.099372 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.099372 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.099399 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.099467 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.099613 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.100376 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.101997 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.102649 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.102724 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.102813 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.103681 (+ 868us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.104391 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2775}
I20260806 06:46:32.105082 13337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 LEARNER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 06:46:32.105309 13288 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 06:46:32.106062 13337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b25ebdad004a49358fe7c1fe808f574f (127.6.57.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.107126 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.107218 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: LEARNER
I20260806 06:46:32.107293 13288 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: attempting to promote NON_VOTER a9f35b8df4714fd1875b666e57026e8a to VOTER
I20260806 06:46:32.107475 13288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } }
I20260806 06:46:32.108026 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.107913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39302)
0806 06:46:32.107956 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.108001 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:42415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:46:32.107944 13481 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3032a4ae73744fd593e028910ee7ed2f. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: NON_VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: true } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.108062 13481 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: LEARNER
I20260806 06:46:32.108458 13337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 LEARNER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 06:46:32.108683 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.108439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39304)
0806 06:46:32.108627 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.108658 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:42415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260806 06:46:32.108778 12691 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f -> Peer b25ebdad004a49358fe7c1fe808f574f (127.6.57.124:42415): Couldn't send request to peer b25ebdad004a49358fe7c1fe808f574f. Status: Network error: Client connection negotiation failed: client connection to 127.6.57.124:42415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:46:32.108843 13482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 06:46:32.109438 13287 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a9f35b8df4714fd1875b666e57026e8a (127.6.57.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.110213 13287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.110298 13287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.110365 13287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.110421 13287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.110707 13337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a9f35b8df4714fd1875b666e57026e8a (127.6.57.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.111595 13481 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.111706 13481 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:46:32.116911 13382 master.cc:552] Master@127.6.57.124:42415 shutdown complete.
I20260806 06:46:32.116968 13382 master_runner.cc:305] Clearing existing system tablet
I20260806 06:46:32.117103 13382 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:32.117165 13382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:32.117779 13382 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260806 06:46:32.118307 13484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.118458 13382 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.118610 13382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:32.118642 13382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "b25ebdad004a49358fe7c1fe808f574f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "ae61f344ab5caecb41296bcc8c95af45"
server_key_iv: "108b346bb270bc40ee8d39d68fd2c9b5"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.119258 13382 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:32.120731 13382 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 06:46:32.120827 13382 log.cc:1196] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 06:46:32.121043 13382 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Deleting consensus metadata
I20260806 06:46:32.121189 13382 master_runner.cc:315] Copying system tablet from 127.6.57.126:43343
I20260806 06:46:32.121778 13382 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: tablet copy: Beginning tablet copy session from remote peer at address 127.6.57.126:43343
I20260806 06:46:32.127938 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.122107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.6.57.126:43343)
0806 06:46:32.122153 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.122156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.122280 (+ 124us) server_negotiation.cc:407] Connection header received
0806 06:46:32.122355 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.122356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.122367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.122411 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.122763 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.123286 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.124004 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.124216 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.124321 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.125318 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.125320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.125320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.125328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.126825 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.127182 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.127183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.127184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.127244 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.127431 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.127432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.127433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.127543 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.127615 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.127767 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.127844 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:32.127962 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.122078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37042)
0806 06:46:32.122205 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.122215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.122295 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.122423 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.122423 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.122635 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.122686 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.122688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.123370 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.123371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.123918 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.123920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.124378 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.124379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.124428 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.124915 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.124920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.125224 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.126923 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.126924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.126924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.127112 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.127316 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.127317 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.127317 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.127375 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.127612 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.127613 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.127660 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.127739 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.127843 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260806 06:46:32.128449 12726 tablet_copy_service.cc:140] P 3032a4ae73744fd593e028910ee7ed2f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b25ebdad004a49358fe7c1fe808f574f ({username='oryx', principal='oryx/127.6.57.124@KRBTEST.COM'} at 127.0.0.1:37042)
I20260806 06:46:32.128521 12726 tablet_copy_service.cc:161] P 3032a4ae73744fd593e028910ee7ed2f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b25ebdad004a49358fe7c1fe808f574f at {username='oryx', principal='oryx/127.6.57.124@KRBTEST.COM'} at 127.0.0.1:37042: session id = b25ebdad004a49358fe7c1fe808f574f-00000000000000000000000000000000
I20260806 06:46:32.128943 12726 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 06:46:32.129405 13382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 06:46:32.130476 13382 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: tablet copy: Starting download of 0 data blocks...
I20260806 06:46:32.130649 13382 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: tablet copy: Starting download of 1 WAL segments...
I20260806 06:46:32.131356 13382 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 06:46:32.132103 12726 tablet_copy_service.cc:342] P 3032a4ae73744fd593e028910ee7ed2f: Request end of tablet copy session b25ebdad004a49358fe7c1fe808f574f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.57.124@KRBTEST.COM'} at 127.0.0.1:37042
I20260806 06:46:32.132174 12726 tablet_copy_service.cc:433] P 3032a4ae73744fd593e028910ee7ed2f: ending tablet copy session b25ebdad004a49358fe7c1fe808f574f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b25ebdad004a49358fe7c1fe808f574f
I20260806 06:46:32.134259 13382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:32.135316 13491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.139631 13382 server_base.cc:1034] running on GCE node
W20260806 06:46:32.142942 13495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.143249 13492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.143404 13382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:32.143585 13382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:32.144703 13382 hybrid_clock.cc:648] HybridClock initialized: now 1785998792144693 us; error 27 us; skew 500 ppm
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:32.146292 13382 init.cc:377] Logged in from keytab as oryx/127.6.57.124@KRBTEST.COM (short username oryx)
I20260806 06:46:32.146593 13382 webserver.cc:460] Webserver started at http://127.6.57.124:35073/ using document root <none> and password file <none>
I20260806 06:46:32.146725 13382 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:32.146799 13382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:32.147367 13382 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.147863 13502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.148008 13382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.148054 13382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "b25ebdad004a49358fe7c1fe808f574f"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "ae61f344ab5caecb41296bcc8c95af45"
server_key_iv: "108b346bb270bc40ee8d39d68fd2c9b5"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.148109 13382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.158407 13382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:32.160022 13382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:32.163144 13382 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.124:42415
I20260806 06:46:32.163537 13382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
I20260806 06:46:32.163825 13554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.124:42415 every 8 connection(s)
I20260806 06:46:32.164397 13555 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:46:32.164847 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Bootstrap starting.
I20260806 06:46:32.167256 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:46:32.167484 13555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: Bootstrap complete.
I20260806 06:46:32.167678 13555 raft_consensus.cc:348] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } }
I20260806 06:46:32.167762 13555 raft_consensus.cc:729] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b25ebdad004a49358fe7c1fe808f574f, State: Initialized, Role: LEARNER
I20260806 06:46:32.167807 13555 consensus_queue.cc:260] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } }
I20260806 06:46:32.167979 13555 sys_catalog.cc:564] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:32.169191 13558 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.169310 13558 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: LEARNER
I20260806 06:46:32.170086 13570 catalog_manager.cc:1259] Loaded cluster ID: b78000ac615f44db8da413ede9ff4642
I20260806 06:46:32.170130 13570 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: loading cluster ID for follower catalog manager: success
I20260806 06:46:32.170908 13570 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: acquiring CA information for follower catalog manager: success
I20260806 06:46:32.171188 13570 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 06:46:32.220021 13479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:32.220294 13479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:32.220331 13479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:32.221989 13479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:32.222056 13479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:32.222093 13479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:32.224140 13479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:41819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=46681
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:32.224376 13479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:32.224601 13479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:32.229905 13580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.229856 13576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.229877 13577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.230177 13479 server_base.cc:1034] running on GCE node
I20260806 06:46:32.230422 13479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:32.230734 13479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:32.231897 13479 hybrid_clock.cc:648] HybridClock initialized: now 1785998792231873 us; error 39 us; skew 500 ppm
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:32.234584 13479 init.cc:377] Logged in from keytab as oryx/127.6.57.65@KRBTEST.COM (short username oryx)
I20260806 06:46:32.235056 13479 webserver.cc:460] Webserver started at http://127.6.57.65:46681/ using document root <none> and password file <none>
I20260806 06:46:32.235278 13479 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:32.235337 13479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:32.236788 13479 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:32.237495 13587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.237695 13479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:32.237805 13479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "3e4edf0ebacaf038dea7d9bb9f1bcfb5"
server_key_iv: "03f8a0e52be74c7c9eeb76d0e671cde8"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.238168 13479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.248227 13479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:32.250587 13479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:32.250741 13479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:32.250973 13479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:32.251263 13479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:32.251312 13479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.251348 13479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:32.251374 13479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.257448 13479 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:41819
I20260806 06:46:32.257537 13704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:41819 every 8 connection(s)
I20260806 06:46:32.257872 13479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:32.266083 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13479
I20260806 06:46:32.266247 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12914
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
I20260806 06:46:32.270797 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:50119 (local address 127.6.57.126:43343)
0806 06:46:32.258650 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.258653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.259165 (+ 512us) server_negotiation.cc:407] Connection header received
0806 06:46:32.260025 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.260026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.260041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.260074 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.260890 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261703 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263067 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263268 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263302 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.267199 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.267201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.267201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.267209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.269151 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269718 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.269720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.269720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.269780 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.270093 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.270094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.270095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.270208 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.270247 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.270633 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.270712 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:46:32.270961 13711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.6.57.65:59137)
0806 06:46:32.258810 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.258853 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.259890 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.260152 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.260155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.260581 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.260787 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.260793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261719 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.261720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.262974 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.262977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263484 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.263540 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.264255 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.264260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.265829 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.269352 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.269353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.269628 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.269859 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.269860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.270005 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.270342 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.270343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.270445 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.270589 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.270740 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":83,"thread_start_us":110,"threads_started":1}
I20260806 06:46:32.271099 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:50119)
0806 06:46:32.259088 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.259097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.259909 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.260154 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.260155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.260575 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.260786 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.260791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261771 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.261772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.262974 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.262977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263398 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.263504 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.264044 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.264066 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.267105 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.269246 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.269254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.269628 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.269856 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.269857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.270014 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.270275 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.270279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.270445 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.270884 (+ 439us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.270968 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":73,"thread_start_us":92,"threads_started":1}
I20260806 06:46:32.271255 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:59137 (local address 127.6.57.124:42415)
0806 06:46:32.259025 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.259027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.259038 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:32.260055 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.260056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.260071 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.260144 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.260903 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261594 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263057 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263367 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263426 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.265934 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.265936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.265937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.265945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.268716 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.269706 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.269707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.269708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.269781 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.270077 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.270078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.270079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.270267 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.270353 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.271092 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.271181 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":50,"threads_started":1}
I20260806 06:46:32.271783 13706 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:32.271865 13706 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.271999 13706 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:32.272387 12706 ts_manager.cc:194] Re-registered known tserver with Master: 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
I20260806 06:46:32.272864 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:50119
I20260806 06:46:32.275254 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:53765 (local address 127.6.57.125:42589)
0806 06:46:32.258942 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.258944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.258954 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:32.259993 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.259995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.260009 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.260056 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.260926 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261730 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.262974 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263276 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263343 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.272284 (+ 8941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.272285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.272285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.272293 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.273933 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.274310 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.274310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.274311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.274358 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.274562 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.274563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.274564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.274714 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.274788 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.275025 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.275111 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:32.275408 13705 heartbeater.cc:347] Connected to a master server at 127.6.57.124:42415
I20260806 06:46:32.275457 13705 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.275527 13705 heartbeater.cc:511] Master 127.6.57.124:42415 requested a full tablet report, sending...
I20260806 06:46:32.275666 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.258606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.6.57.65:53765)
0806 06:46:32.258878 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.258889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.259890 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.260151 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.260155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.260580 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.260787 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.260793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.261814 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.261816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.262901 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.262905 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.263364 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.263365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.263504 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.264096 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.264101 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.272209 (+ 8108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.274014 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.274015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.274015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.274234 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.274414 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.274415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.274415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.274507 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.274840 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.274841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.274902 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.275365 (+ 463us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.275498 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":93,"thread_start_us":114,"threads_started":1}
I20260806 06:46:32.275854 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:37667
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:43343
--webserver_port=41949
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:46:32.275924 13519 ts_manager.cc:194] Registered new tserver with Master: 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
I20260806 06:46:32.276134 13707 heartbeater.cc:347] Connected to a master server at 127.6.57.125:42589
I20260806 06:46:32.276182 13707 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.276233 13707 heartbeater.cc:511] Master 127.6.57.125:42589 requested a full tablet report, sending...
I20260806 06:46:32.276552 13327 ts_manager.cc:194] Registered new tserver with Master: 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
W20260806 06:46:32.358546 13714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:32.358739 13714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:32.358762 13714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:32.359876 13714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:32.359915 13714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:32.359961 13714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:32.361510 13714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:37667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=41949
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:32.361747 13714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:32.361968 13714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:32.366736 13722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.366767 13720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.366756 13719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.366954 13714 server_base.cc:1034] running on GCE node
I20260806 06:46:32.367187 13714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:32.367415 13714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:32.368593 13714 hybrid_clock.cc:648] HybridClock initialized: now 1785998792368570 us; error 33 us; skew 500 ppm
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:32.371109 13714 init.cc:377] Logged in from keytab as oryx/127.6.57.66@KRBTEST.COM (short username oryx)
I20260806 06:46:32.371510 13714 webserver.cc:460] Webserver started at http://127.6.57.66:41949/ using document root <none> and password file <none>
I20260806 06:46:32.371735 13714 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:32.371812 13714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:32.373052 13714 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:32.373596 13729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.373773 13714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.373870 13714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "ed661da5b2654ac2afe7e25d3be8237b"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "4c72cdb403dd9d65b7ab3ccc6fdf4188"
server_key_iv: "e5c3ec938232870779d902ba9367647d"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.374197 13714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.391222 13714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:32.392899 13714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:32.393051 13714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:32.393256 13714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:32.393533 13714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:32.393584 13714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.393622 13714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:32.393647 13714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.399552 13714 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:37667
I20260806 06:46:32.399593 13845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:37667 every 8 connection(s)
I20260806 06:46:32.399940 13714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:32.401021 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13714
I20260806 06:46:32.401204 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13052
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
I20260806 06:46:32.409493 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:46355
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:43343
--webserver_port=36711
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:46:32.413025 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:57689 (local address 127.6.57.124:42415)
0806 06:46:32.400992 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.400994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.401067 (+ 73us) server_negotiation.cc:407] Connection header received
0806 06:46:32.403357 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.403358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.403369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.403396 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.404157 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.404950 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406120 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406523 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406571 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.409832 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.409833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.409833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.409839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.411557 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.411960 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.411961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.411961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.412024 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.412354 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.412355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.412355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.412470 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.412562 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.412840 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.412934 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:46:32.413205 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.6.57.66:57689)
0806 06:46:32.400938 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.400980 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.403184 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.403835 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.403835 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.403990 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.404046 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.404048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.405053 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.405054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.406007 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.406010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406654 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.406732 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.407159 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.407164 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.409744 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.411648 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.411654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.411656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.411862 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.412079 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.412080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.412080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.412249 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.412556 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.412564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.412696 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.412806 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.412958 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":107,"threads_started":1}
I20260806 06:46:32.413687 13846 heartbeater.cc:347] Connected to a master server at 127.6.57.124:42415
I20260806 06:46:32.413806 13846 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.414036 13846 heartbeater.cc:511] Master 127.6.57.124:42415 requested a full tablet report, sending...
I20260806 06:46:32.414390 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:39739 (local address 127.6.57.126:43343)
0806 06:46:32.400782 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.400785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.401147 (+ 362us) server_negotiation.cc:407] Connection header received
0806 06:46:32.401768 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.401770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.401784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.401812 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.403145 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.404134 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.405925 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406240 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406280 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.409692 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.409694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.409694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.409701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.413038 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413470 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.413471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.413471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.413553 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413795 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.413796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.413797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.413925 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.413991 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.414206 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.414293 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:46:32.414515 13519 ts_manager.cc:194] Registered new tserver with Master: ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667)
I20260806 06:46:32.414774 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:39739)
0806 06:46:32.401087 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.401096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.401657 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.402405 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.402410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.402789 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.403040 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.403046 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.404413 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.404414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.405830 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.405833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406365 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.406546 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.407017 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.407039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.409564 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.413140 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.413142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.413374 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.413630 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.413631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.413699 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.414041 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.414042 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.414084 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.414562 (+ 478us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.414665 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260806 06:46:32.415107 13847 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:32.415113 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:35837 (local address 127.6.57.125:42589)
0806 06:46:32.400846 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.400848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.401196 (+ 348us) server_negotiation.cc:407] Connection header received
0806 06:46:32.403257 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.403258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.403271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.403299 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.405703 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406416 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.407386 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.407702 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.407764 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.409915 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.409916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.409917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.409925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.411738 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413730 (+ 1992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.413743 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.413744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.413805 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.414027 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.414027 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.414028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.414159 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.414201 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.414878 (+ 677us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.414973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:46:32.415158 13847 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.415231 13847 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:32.415542 12706 ts_manager.cc:194] Re-registered known tserver with Master: ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667)
I20260806 06:46:32.415716 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.400659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.6.57.66:35837)
0806 06:46:32.401117 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.401129 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.403137 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.403454 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.403454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.403654 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.403741 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.403743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.406540 (+ 2797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.406541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.407264 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.407266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.407843 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.407844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.407917 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.408298 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.408303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.409818 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.413467 (+ 3649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.413469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.413670 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.413875 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.413876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.413876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.413957 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.414263 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.414264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.414318 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.415233 (+ 915us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.415357 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":287,"thread_start_us":63,"threads_started":1}
I20260806 06:46:32.416056 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:39739
I20260806 06:46:32.416090 13848 heartbeater.cc:347] Connected to a master server at 127.6.57.125:42589
I20260806 06:46:32.416136 13848 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.416215 13848 heartbeater.cc:511] Master 127.6.57.125:42589 requested a full tablet report, sending...
I20260806 06:46:32.416517 13327 ts_manager.cc:194] Registered new tserver with Master: ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667)
W20260806 06:46:32.496742 13854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:32.496930 13854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:32.496954 13854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:32.498160 13854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:32.498214 13854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:32.498240 13854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:32.499759 13854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:46355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=36711
--tserver_master_addrs=127.6.57.126:43343,127.6.57.125:42589,127.6.57.124:42415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:32.499993 13854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:32.500221 13854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:32.505162 13863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:32.505147 13859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.505221 13854 server_base.cc:1034] running on GCE node
W20260806 06:46:32.505131 13861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:32.505491 13854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:32.505721 13854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:32.506878 13854 hybrid_clock.cc:648] HybridClock initialized: now 1785998792506859 us; error 29 us; skew 500 ppm
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:32.509434 13854 init.cc:377] Logged in from keytab as oryx/127.6.57.67@KRBTEST.COM (short username oryx)
I20260806 06:46:32.509810 13854 webserver.cc:460] Webserver started at http://127.6.57.67:36711/ using document root <none> and password file <none>
I20260806 06:46:32.509984 13854 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:32.510044 13854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:32.511358 13854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:32.511912 13870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.512118 13854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:32.512220 13854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "25bda12ce5b54fc28ddb550965e56938"
format_stamp: "Formatted at 2026-08-06 06:46:31 on dist-test-slave-5l3k"
server_key: "6bb8d4dc2dc156c08aa71c06606b2667"
server_key_iv: "8f395734bacd91706db9ec334b0d865e"
server_key_version: "encryptionkey@0"
I20260806 06:46:32.512543 13854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:32.523949 13854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:32.525642 13854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:32.525746 13854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:32.525911 13854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:32.526196 13854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:32.526230 13854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.526252 13854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:32.526263 13854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:32.532135 13854 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:46355
I20260806 06:46:32.532203 13985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:46355 every 8 connection(s)
I20260806 06:46:32.532550 13854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:32.537225 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 13854
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.125@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.124@KRBTEST.COM
Aug 06 06:46:32 dist-test-slave-5l3k krb5kdc[12658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998792, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:32.544210 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:58909 (local address 127.6.57.125:42589)
0806 06:46:32.533273 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.533280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.533520 (+ 240us) server_negotiation.cc:407] Connection header received
0806 06:46:32.534105 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.534107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.534122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.534208 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.535089 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535810 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537038 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537304 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537364 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.539966 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.539967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.539967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.539975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.542768 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543160 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.543161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.543162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.543219 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543454 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.543454 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.543455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.543570 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.543621 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.544019 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.544112 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 06:46:32.544338 13992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.6.57.67:44841)
0806 06:46:32.534160 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.534175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.534266 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.534442 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.534444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.534746 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.534927 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.534932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535905 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.535906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.536957 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.536958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537485 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.537579 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.538694 (+ 1115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.538700 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.540469 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.542668 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.542674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.542677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.543018 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.543250 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.543251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.543391 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.543704 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.543705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.543810 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.543911 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.544096 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":8,"thread_start_us":72,"threads_started":1}
I20260806 06:46:32.544683 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:44841 (local address 127.6.57.124:42415)
0806 06:46:32.533631 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.533632 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.534238 (+ 606us) server_negotiation.cc:407] Connection header received
0806 06:46:32.534343 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.534344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.534354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.534410 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.535037 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535808 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537069 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537352 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537421 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.540553 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.540555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.540555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.540563 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.542563 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543109 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.543110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.543111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.543173 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543467 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.543468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.543469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.543613 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.543675 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.544469 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.544566 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:32.544867 13991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.6.57.67:58909)
0806 06:46:32.533392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.533437 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.533951 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.534297 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.534301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.534967 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.535026 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.535028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535905 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.535906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.536950 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.536952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537414 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.537474 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.538053 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.538064 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.539862 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.542856 (+ 2994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.542857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.542857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.543087 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.543285 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.543286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.543390 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.543643 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.543649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.543809 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.543911 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.544754 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":754,"thread_start_us":100,"threads_started":1}
I20260806 06:46:32.544999 13990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:40393)
0806 06:46:32.533392 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.533437 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.533953 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.535099 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.535100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.535195 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:32.535230 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.535231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535889 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.535890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.536749 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.536751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537228 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.537436 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.538538 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:32.538544 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:32.541862 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:32.543812 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.543813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.543813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.544047 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.544265 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:32.544266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:32.544266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:32.544330 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:32.544673 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:32.544673 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:32.544723 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.544793 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.544886 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":60,"spinlock_wait_cycles":7552,"thread_start_us":87,"threads_started":1}
I20260806 06:46:32.545372 13986 heartbeater.cc:347] Connected to a master server at 127.6.57.124:42415
I20260806 06:46:32.545406 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.533189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:40393 (local address 127.6.57.126:43343)
0806 06:46:32.533237 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.533239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.533521 (+ 282us) server_negotiation.cc:407] Connection header received
0806 06:46:32.534111 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.534113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.534126 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.534197 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.535306 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.535790 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.536841 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.537105 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.537161 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.541969 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.541970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.541971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.541978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.543752 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.544127 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.544128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.544128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.544194 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.544451 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.544452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.544452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.544572 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.544646 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.545202 (+ 556us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.545298 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:32.545449 13986 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.545493 13988 heartbeater.cc:347] Connected to a master server at 127.6.57.125:42589
I20260806 06:46:32.545531 13988 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.545647 13986 heartbeater.cc:511] Master 127.6.57.124:42415 requested a full tablet report, sending...
I20260806 06:46:32.545704 13987 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43343
I20260806 06:46:32.545735 13987 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:32.545782 13987 heartbeater.cc:511] Master 127.6.57.126:43343 requested a full tablet report, sending...
I20260806 06:46:32.545647 13988 heartbeater.cc:511] Master 127.6.57.125:42589 requested a full tablet report, sending...
I20260806 06:46:32.546046 12706 ts_manager.cc:194] Re-registered known tserver with Master: 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:32.546123 13327 ts_manager.cc:194] Registered new tserver with Master: 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:32.546411 13519 ts_manager.cc:194] Registered new tserver with Master: 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:32.546561 12706 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:40393
I20260806 06:46:32.551542 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.538413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.6.57.124:42415)
0806 06:46:32.542146 (+ 3733us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.542148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.542154 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:46:32.542176 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.542177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.542200 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.542999 (+ 799us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.543034 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.543728 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.546693 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.546978 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.547290 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.548430 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.548431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.548431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.548437 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.550077 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.550454 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.550455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.550455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.550541 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.550767 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.550768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.550768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.550903 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.551015 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.551128 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.551477 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3710,"thread_start_us":54,"threads_started":1}
I20260806 06:46:32.552139 14000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.538556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37050 (local address 127.6.57.126:43343)
0806 06:46:32.538718 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.538719 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.539113 (+ 394us) server_negotiation.cc:407] Connection header received
0806 06:46:32.539168 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.539169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.539186 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.539261 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.540006 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.540833 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.543824 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.544149 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.548049 (+ 3900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.548088 (+ 39us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.548089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.548089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.548092 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.550759 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.551306 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.551308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.551308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.551382 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.551599 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.551601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.551601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.551742 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.551848 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.551953 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.552037 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260806 06:46:32.553640 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.538953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35166 (local address 127.6.57.125:42589)
0806 06:46:32.539210 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.539212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.545832 (+ 6620us) server_negotiation.cc:407] Connection header received
0806 06:46:32.545891 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.545892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.545905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.546835 (+ 930us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.546866 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.547597 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.548661 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.548986 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.549058 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.550583 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.550584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.550585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.550592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.552531 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.552906 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.552907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.552908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.552973 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.553165 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.553166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.553166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.553298 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.553395 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.553520 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.553580 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":80,"threads_started":1}
I20260806 06:46:32.553651 12706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:32.555696 12706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:32.563266 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:37667 (local address 127.0.0.1:53498)
0806 06:46:32.559348 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.559359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.559412 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.559604 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.559605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.559607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.559640 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.559642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.560899 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.560900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.562068 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.562071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.563047 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.563105 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.563109 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.563138 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.563176 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":68,"thread_start_us":79,"threads_started":1}
I20260806 06:46:32.563407 14009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:46355 (local address 127.0.0.1:43568)
0806 06:46:32.559355 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.559377 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.559413 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.560266 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.560267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.560269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.560305 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.560307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.561352 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.561353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.562342 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.562344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.563195 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.563252 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.563256 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.563298 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.563340 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260806 06:46:32.563434 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.6.57.67:46355)
0806 06:46:32.559925 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.559929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.559939 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:32.560134 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.560136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.560151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.560261 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.560367 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.561274 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.562430 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.563106 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563230 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.563310 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.563335 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.563367 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260806 06:46:32.563679 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.6.57.66:37667)
0806 06:46:32.559457 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.559461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.559470 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:32.559508 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.559509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.559521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.559608 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.559704 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.560800 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.562175 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.562932 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563423 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.563496 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.563522 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.563570 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260806 06:46:32.563938 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41819 (local address 127.0.0.1:36962)
0806 06:46:32.559284 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.559296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.559350 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.559846 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.559847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.559849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.559892 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.559894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.560897 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.560898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.562323 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.562325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.563679 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.563822 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.563831 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.563870 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.563898 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":53}
I20260806 06:46:32.564073 13918 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa6e09359b24f1495e95260877aad2d (DEFAULT_TABLE table=test-table [id=3feb48d696324a74a4ccce0ff8e31594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:32.564217 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.559505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.6.57.65:41819)
0806 06:46:32.559674 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.559678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.559688 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:32.559728 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.559730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.559744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.559840 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.559971 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.560815 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.562424 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.563165 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.563278 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.564105 (+ 827us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.564135 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.564165 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260806 06:46:32.564256 13777 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa6e09359b24f1495e95260877aad2d (DEFAULT_TABLE table=test-table [id=3feb48d696324a74a4ccce0ff8e31594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:32.564442 13918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa6e09359b24f1495e95260877aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:32.564579 13777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa6e09359b24f1495e95260877aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:32.567056 13636 tablet_service.cc:1467] Processing CreateTablet for tablet 2aa6e09359b24f1495e95260877aad2d (DEFAULT_TABLE table=test-table [id=3feb48d696324a74a4ccce0ff8e31594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:32.567399 13636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aa6e09359b24f1495e95260877aad2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:32.568514 14013 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Bootstrap starting.
I20260806 06:46:32.569309 14013 tablet_bootstrap.cc:654] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:32.569677 14013 log.cc:824] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:32.568300 14014 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Bootstrap starting.
I20260806 06:46:32.570416 14013 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: No bootstrap required, opened a new log
I20260806 06:46:32.570520 14013 ts_tablet_manager.cc:1397] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:32.570581 14014 tablet_bootstrap.cc:654] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:32.571014 14016 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Bootstrap starting.
I20260806 06:46:32.570935 14014 log.cc:824] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:32.571702 14016 tablet_bootstrap.cc:654] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:32.571765 14014 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: No bootstrap required, opened a new log
I20260806 06:46:32.571831 14014 ts_tablet_manager.cc:1397] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 06:46:32.572113 14016 log.cc:824] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:32.573026 14016 tablet_bootstrap.cc:492] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: No bootstrap required, opened a new log
I20260806 06:46:32.573089 14016 ts_tablet_manager.cc:1397] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:32.573434 14013 raft_consensus.cc:348] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.573593 14013 raft_consensus.cc:374] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:32.573632 14013 raft_consensus.cc:729] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed661da5b2654ac2afe7e25d3be8237b, State: Initialized, Role: FOLLOWER
I20260806 06:46:32.573755 14013 consensus_queue.cc:260] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.574007 14013 ts_tablet_manager.cc:1428] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:32.574137 13847 heartbeater.cc:503] Master 127.6.57.126:43343 was elected leader, sending a full tablet report...
I20260806 06:46:32.575263 14014 raft_consensus.cc:348] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.575400 14014 raft_consensus.cc:374] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:32.575440 14014 raft_consensus.cc:729] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bda12ce5b54fc28ddb550965e56938, State: Initialized, Role: FOLLOWER
I20260806 06:46:32.575553 14014 consensus_queue.cc:260] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.575832 13987 heartbeater.cc:503] Master 127.6.57.126:43343 was elected leader, sending a full tablet report...
I20260806 06:46:32.576030 14016 raft_consensus.cc:348] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.576476 14014 ts_tablet_manager.cc:1428] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260806 06:46:32.576536 14016 raft_consensus.cc:374] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:32.576599 14016 raft_consensus.cc:729] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168ed8a28d2a4214ab7e36d9b6f8a68d, State: Initialized, Role: FOLLOWER
I20260806 06:46:32.576707 14016 consensus_queue.cc:260] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.576968 14016 ts_tablet_manager.cc:1428] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 06:46:32.577361 13706 heartbeater.cc:503] Master 127.6.57.126:43343 was elected leader, sending a full tablet report...
I20260806 06:46:32.578637 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.574932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.6.57.124:42415)
0806 06:46:32.574997 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.574999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.575010 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:32.575040 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.575041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.575062 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.575138 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.575314 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.576024 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.577469 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.578064 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.578125 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.578193 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.578534 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.578579 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:32.578764 14009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.574830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39330)
0806 06:46:32.574932 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.574944 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.574981 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.575187 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.575188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.575190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.575229 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.575231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.576197 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.576198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.577347 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.577350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.578227 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.578228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.578291 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.578295 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.578347 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.578392 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 06:46:32.579103 13529 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260806 06:46:32.579864 13558 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3032a4ae73744fd593e028910ee7ed2f. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.579946 13558 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: LEARNER
I20260806 06:46:32.645133 14019 raft_consensus.cc:482] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:32.645252 14019 raft_consensus.cc:504] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.645668 14019 leader_election.cc:288] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819), 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:32.648797 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.645932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:48107 (local address 127.6.57.65:41819)
0806 06:46:32.645986 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.645988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.645999 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:32.646127 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.646129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.646145 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.646205 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.646457 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.646984 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648007 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.648426 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648511 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.648522 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.648692 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.648732 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:46:32.648798 13853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.645844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41819 (local address 127.6.57.66:48107)
0806 06:46:32.645901 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.645913 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.645985 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.646217 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.646218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.646222 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.646344 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.646345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.647121 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.647122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.647929 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.647931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.648509 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.648598 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.648602 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.648681 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.648724 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":17}
I20260806 06:46:32.648854 14012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.646098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:60383 (local address 127.6.57.67:46355)
0806 06:46:32.646163 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.646165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.646173 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:32.646200 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.646201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.646213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.646274 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:32.646418 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.646962 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648085 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.648475 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648558 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.648567 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.648750 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.648787 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 06:46:32.648929 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.645968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:46355 (local address 127.6.57.66:60383)
0806 06:46:32.646015 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:32.646025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:32.646064 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:32.646284 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:32.646284 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:32.646287 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:32.646342 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.646344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.647044 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.647045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.648014 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:32.648017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.648559 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.648560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:32.648659 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.648662 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:32.648741 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:32.648780 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":40}
I20260806 06:46:32.649250 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa6e09359b24f1495e95260877aad2d" candidate_uuid: "ed661da5b2654ac2afe7e25d3be8237b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bda12ce5b54fc28ddb550965e56938" is_pre_election: true
I20260806 06:46:32.649245 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa6e09359b24f1495e95260877aad2d" candidate_uuid: "ed661da5b2654ac2afe7e25d3be8237b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" is_pre_election: true
I20260806 06:46:32.649401 13656 raft_consensus.cc:2454] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed661da5b2654ac2afe7e25d3be8237b in term 0.
I20260806 06:46:32.649401 13938 raft_consensus.cc:2454] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed661da5b2654ac2afe7e25d3be8237b in term 0.
I20260806 06:46:32.649621 13731 leader_election.cc:302] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bda12ce5b54fc28ddb550965e56938, ed661da5b2654ac2afe7e25d3be8237b; no voters:
I20260806 06:46:32.649758 14019 raft_consensus.cc:2781] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:32.649834 14019 raft_consensus.cc:482] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:32.649868 14019 raft_consensus.cc:3037] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:32.650683 14019 raft_consensus.cc:504] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.650817 14019 leader_election.cc:288] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [CANDIDATE]: Term 1 election: Requested vote from peers 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819), 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355)
I20260806 06:46:32.650969 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa6e09359b24f1495e95260877aad2d" candidate_uuid: "ed661da5b2654ac2afe7e25d3be8237b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bda12ce5b54fc28ddb550965e56938"
I20260806 06:46:32.650993 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aa6e09359b24f1495e95260877aad2d" candidate_uuid: "ed661da5b2654ac2afe7e25d3be8237b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
I20260806 06:46:32.651046 13938 raft_consensus.cc:3037] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:32.651095 13656 raft_consensus.cc:3037] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Advancing to term 1
W20260806 06:46:32.651459 13849 tablet.cc:2369] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:32.651851 13938 raft_consensus.cc:2454] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed661da5b2654ac2afe7e25d3be8237b in term 1.
I20260806 06:46:32.652037 13731 leader_election.cc:302] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bda12ce5b54fc28ddb550965e56938, ed661da5b2654ac2afe7e25d3be8237b; no voters:
I20260806 06:46:32.652084 13656 raft_consensus.cc:2454] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed661da5b2654ac2afe7e25d3be8237b in term 1.
I20260806 06:46:32.652144 14019 raft_consensus.cc:2781] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:32.652310 14019 raft_consensus.cc:686] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 1 LEADER]: Becoming Leader. State: Replica: ed661da5b2654ac2afe7e25d3be8237b, State: Running, Role: LEADER
I20260806 06:46:32.652416 14019 consensus_queue.cc:237] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } }
I20260806 06:46:32.653230 12706 catalog_manager.cc:5495] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b reported cstate change: term changed from 0 to 1, leader changed from <none> to ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66). New cstate: current_term: 1 leader_uuid: "ed661da5b2654ac2afe7e25d3be8237b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:32.653925 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: NON_VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: true } } }
I20260806 06:46:32.654028 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.654568 13288 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: attempting to promote NON_VOTER b25ebdad004a49358fe7c1fe808f574f to VOTER
I20260806 06:46:32.654703 13288 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } }
I20260806 06:46:32.655159 13337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:46:32.655231 13529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 LEARNER]: Refusing update from remote peer 3032a4ae73744fd593e028910ee7ed2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260806 06:46:32.655480 13482 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260806 06:46:32.655596 14031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260806 06:46:32.656104 13482 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b25ebdad004a49358fe7c1fe808f574f (127.6.57.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.656441 13529 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b25ebdad004a49358fe7c1fe808f574f (127.6.57.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.656989 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.657011 14030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.657078 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.657088 14030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.657218 14030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.657269 14030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:32.657351 13337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b25ebdad004a49358fe7c1fe808f574f (127.6.57.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.657481 13558 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.657573 13558 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:46:32.658195 14032 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3032a4ae73744fd593e028910ee7ed2f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:32.658305 14032 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:46:32.679885 14010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.677365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.6.57.66:37667)
0806 06:46:32.677412 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.677414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.677424 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:32.677502 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.677503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.677562 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.677634 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:32.677829 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.678280 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.679120 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.679319 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.679394 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.679570 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:32.679668 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:32.679755 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.679774 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.679795 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:32.681638 13656 raft_consensus.cc:1261] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Refusing update from remote peer ed661da5b2654ac2afe7e25d3be8237b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:32.681635 13938 raft_consensus.cc:1261] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Refusing update from remote peer ed661da5b2654ac2afe7e25d3be8237b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:32.681998 14026 consensus_queue.cc:1035] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [LEADER]: Connected to new peer: Peer: permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:32.682168 14019 consensus_queue.cc:1035] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [LEADER]: Connected to new peer: Peer: permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:32.686044 14036 mvcc.cc:205] Tried to move back new op lower bound from 7315451054821097472 to 7315451054704910336. Current Snapshot: MvccSnapshot[applied={T|T < 7315451054821097472}]
I20260806 06:46:32.686183 14035 mvcc.cc:205] Tried to move back new op lower bound from 7315451054821097472 to 7315451054704910336. Current Snapshot: MvccSnapshot[applied={T|T < 7315451054821097472}]
I20260806 06:46:32.688413 12705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:32.688508 12705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:32.689741 12705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3032a4ae73744fd593e028910ee7ed2f: Sending DeleteTablet for 3 replicas of tablet 2aa6e09359b24f1495e95260877aad2d
I20260806 06:46:32.690035 13777 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa6e09359b24f1495e95260877aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:32 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53498
I20260806 06:46:32.690063 13636 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa6e09359b24f1495e95260877aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:32 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:36962
I20260806 06:46:32.690047 13918 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aa6e09359b24f1495e95260877aad2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:32 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43568
I20260806 06:46:32.690344 14043 tablet_replica.cc:330] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: stopping tablet replica
I20260806 06:46:32.690439 14043 raft_consensus.cc:2229] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:32.690568 14045 tablet_replica.cc:330] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: stopping tablet replica
I20260806 06:46:32.690800 14045 raft_consensus.cc:2229] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:32.691239 14045 raft_consensus.cc:2258] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:32.691329 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 12673
I20260806 06:46:32.692487 14043 raft_consensus.cc:2258] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:32.692926 14045 ts_tablet_manager.cc:1905] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:32.692963 14043 ts_tablet_manager.cc:1905] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:32.694314 14045 ts_tablet_manager.cc:1918] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:32.694387 14045 log.cc:1196] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/2aa6e09359b24f1495e95260877aad2d
I20260806 06:46:32.694689 14045 ts_tablet_manager.cc:1939] T 2aa6e09359b24f1495e95260877aad2d P 25bda12ce5b54fc28ddb550965e56938: Deleting consensus metadata
I20260806 06:46:32.694998 14043 ts_tablet_manager.cc:1918] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:32.695062 14043 log.cc:1196] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/2aa6e09359b24f1495e95260877aad2d
I20260806 06:46:32.695303 14043 ts_tablet_manager.cc:1939] T 2aa6e09359b24f1495e95260877aad2d P ed661da5b2654ac2afe7e25d3be8237b: Deleting consensus metadata
I20260806 06:46:32.695686 14044 tablet_replica.cc:330] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: stopping tablet replica
I20260806 06:46:32.695778 14044 raft_consensus.cc:2229] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:32.695873 14044 raft_consensus.cc:2258] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:32.698933 14044 ts_tablet_manager.cc:1905] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:46:32.699718 13871 connection.cc:535] server connection from 127.0.0.1:43568 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:46:32.700069 13733 connection.cc:535] server connection from 127.0.0.1:53498 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:46:32.701022 14044 ts_tablet_manager.cc:1918] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:32.701103 14044 log.cc:1196] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/2aa6e09359b24f1495e95260877aad2d
I20260806 06:46:32.701380 14044 ts_tablet_manager.cc:1939] T 2aa6e09359b24f1495e95260877aad2d P 168ed8a28d2a4214ab7e36d9b6f8a68d: Deleting consensus metadata
W20260806 06:46:32.701731 13591 connection.cc:441] server connection from 127.0.0.1:36962 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36962 (request call id 1) could send its response
I20260806 06:46:32.707778 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.690707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.6.57.124:42415)
0806 06:46:32.690746 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.690748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.698738 (+ 7990us) server_negotiation.cc:407] Connection header received
0806 06:46:32.698846 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.698847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.698858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.698887 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.699677 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.700451 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.701530 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.701816 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.701856 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.704090 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.704091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.704091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.704098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.705789 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.706693 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.706695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.706695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.706756 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.706953 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.706954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.706954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.707071 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.707167 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.707372 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.707446 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:32.710621 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.702238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.6.57.125:42589)
0806 06:46:32.702280 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.702283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.702693 (+ 410us) server_negotiation.cc:407] Connection header received
0806 06:46:32.702811 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.702813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.702826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.702859 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.703679 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.704439 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.705493 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.705796 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.705834 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.707420 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.707422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.707422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.707429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.709310 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.709759 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.709760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.709760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.709821 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.710020 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.710021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.710021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.710166 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.710284 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.710418 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.710525 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:32.710723 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:32.702439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.6.57.124:42415)
0806 06:46:32.702510 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:32.702512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:32.702912 (+ 400us) server_negotiation.cc:407] Connection header received
0806 06:46:32.703035 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:32.703036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:32.703046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:32.703078 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:32.703795 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.704494 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.705638 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:32.705914 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:32.705950 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:32.707463 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:32.707464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:32.707464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:32.707470 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:32.709404 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.709831 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.709832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.709832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.709889 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:32.710089 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:32.710090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:32.710090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:32.710210 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:32.710324 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:32.710541 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:46:32.710623 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:33.686569 14054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:33.686211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:49139)
0806 06:46:33.686356 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:33.686446 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
W20260806 06:46:33.686863 13706 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:43343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:33.687088 14055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:33.686755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:36935)
0806 06:46:33.686877 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:33.687039 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
W20260806 06:46:33.687278 13847 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:43343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:33.687387 14056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:33.687082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:52727)
0806 06:46:33.687247 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:33.687310 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":53,"threads_started":1}
W20260806 06:46:33.687568 13987 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:43343 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.190658 14058 raft_consensus.cc:482] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3032a4ae73744fd593e028910ee7ed2f)
I20260806 06:46:34.190659 14057 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3032a4ae73744fd593e028910ee7ed2f)
I20260806 06:46:34.190819 14057 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } }
I20260806 06:46:34.190970 14058 raft_consensus.cc:504] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } }
I20260806 06:46:34.191203 14057 leader_election.cc:288] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343), b25ebdad004a49358fe7c1fe808f574f (127.6.57.124:42415)
I20260806 06:46:34.191339 14058 leader_election.cc:288] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343), a9f35b8df4714fd1875b666e57026e8a (127.6.57.125:42589)
I20260806 06:46:34.191726 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37508)
0806 06:46:34.191634 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.191680 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":49,"threads_started":1}
W20260806 06:46:34.191903 13503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.192030 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37512)
0806 06:46:34.191891 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.191968 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":39,"threads_started":1}
W20260806 06:46:34.192157 13310 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.192340 14061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.192252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37514)
0806 06:46:34.192289 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.192313 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:46:34.192420 13503 leader_election.cc:334] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343): Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.192770 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.192660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37526)
0806 06:46:34.192714 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.192745 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:46:34.192845 13310 leader_election.cc:334] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343): Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.194501 14063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.6.57.124:42415)
0806 06:46:34.191767 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.191770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.191778 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:34.191804 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.191805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.191816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.192062 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.192103 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.192845 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.193780 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.194142 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.194219 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.194232 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.194384 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.194420 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
I20260806 06:46:34.194478 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:42415 (local address 127.0.0.1:39352)
0806 06:46:34.191599 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.191611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.191669 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.191911 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.191913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.191915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.191979 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.191982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.192941 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.192942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.193697 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.193699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.194235 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.194237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.194300 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.194304 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.194366 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.194411 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":48,"threads_started":1}
I20260806 06:46:34.194716 13529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9f35b8df4714fd1875b666e57026e8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b25ebdad004a49358fe7c1fe808f574f" is_pre_election: true
I20260806 06:46:34.194787 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.6.57.125:42589)
0806 06:46:34.191795 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.191796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.191907 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:46:34.192058 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.192059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.192103 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.192152 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.192415 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.193098 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.194011 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.194377 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.194458 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.194470 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.194690 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.194731 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":45,"threads_started":1}
I20260806 06:46:34.194818 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.191530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:42589 (local address 127.0.0.1:35190)
0806 06:46:34.191842 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.191853 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.191936 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.192243 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.192244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.192247 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.192303 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.192305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.193190 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.193191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.193919 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.193921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.194472 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.194474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.194584 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.194589 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.194646 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.194696 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":35,"threads_started":1}
I20260806 06:46:34.194840 13529 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9f35b8df4714fd1875b666e57026e8a in term 1.
I20260806 06:46:34.195001 13311 leader_election.cc:302] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9f35b8df4714fd1875b666e57026e8a, b25ebdad004a49358fe7c1fe808f574f; no voters: 3032a4ae73744fd593e028910ee7ed2f
I20260806 06:46:34.195117 14057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 06:46:34.195159 14057 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3032a4ae73744fd593e028910ee7ed2f)
I20260806 06:46:34.195183 14057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:46:34.195217 13337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b25ebdad004a49358fe7c1fe808f574f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a9f35b8df4714fd1875b666e57026e8a" is_pre_election: true
I20260806 06:46:34.195866 14057 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } }
I20260806 06:46:34.195997 13337 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b25ebdad004a49358fe7c1fe808f574f in current term 2: Already voted for candidate a9f35b8df4714fd1875b666e57026e8a in this term.
I20260806 06:46:34.196023 14057 leader_election.cc:288] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 election: Requested vote from peers 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343), b25ebdad004a49358fe7c1fe808f574f (127.6.57.124:42415)
I20260806 06:46:34.196239 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.196091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37530)
0806 06:46:34.196184 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.196214 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 06:46:34.196205 13529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9f35b8df4714fd1875b666e57026e8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b25ebdad004a49358fe7c1fe808f574f"
I20260806 06:46:34.196210 13506 leader_election.cc:302] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b25ebdad004a49358fe7c1fe808f574f; no voters: 3032a4ae73744fd593e028910ee7ed2f, a9f35b8df4714fd1875b666e57026e8a
I20260806 06:46:34.196310 13529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:46:34.196555 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.196455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37546)
0806 06:46:34.196502 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.196536 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 06:46:34.196635 13310 leader_election.cc:334] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343): Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
I20260806 06:46:34.197084 13529 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9f35b8df4714fd1875b666e57026e8a in term 2.
I20260806 06:46:34.197268 14058 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:46:34.197273 13311 leader_election.cc:302] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9f35b8df4714fd1875b666e57026e8a, b25ebdad004a49358fe7c1fe808f574f; no voters: 3032a4ae73744fd593e028910ee7ed2f
I20260806 06:46:34.197377 14057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:46:34.197531 14057 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [term 2 LEADER]: Becoming Leader. State: Replica: a9f35b8df4714fd1875b666e57026e8a, State: Running, Role: LEADER
I20260806 06:46:34.197629 14057 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } }
I20260806 06:46:34.197991 14069 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f35b8df4714fd1875b666e57026e8a. Latest consensus state: current_term: 2 leader_uuid: "a9f35b8df4714fd1875b666e57026e8a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:34.198071 14069 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:34.198230 14071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:34.198670 14071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:34.198768 14071 catalog_manager.cc:1259] Loaded cluster ID: b78000ac615f44db8da413ede9ff4642
I20260806 06:46:34.198800 14071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:34.198968 14071 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:34.199070 14071 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Loaded TSK: 0
I20260806 06:46:34.199206 14071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:34.246668 13327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:34.248471 13327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:34.249595 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.249455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37556)
0806 06:46:34.249525 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.249564 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 06:46:34.249589 13529 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [term 2 FOLLOWER]: Refusing update from remote peer a9f35b8df4714fd1875b666e57026e8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 06:46:34.249919 14069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 06:46:34.250015 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.249907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.0.0.1:37560)
0806 06:46:34.249962 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.249990 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 06:46:34.250135 13310 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a -> Peer 3032a4ae73744fd593e028910ee7ed2f (127.6.57.126:43343): Couldn't send request to peer 3032a4ae73744fd593e028910ee7ed2f. Status: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 06:46:34.250641 14058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f35b8df4714fd1875b666e57026e8a. Latest consensus state: current_term: 2 leader_uuid: "a9f35b8df4714fd1875b666e57026e8a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:34.250751 14058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:46:34.251155 14057 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a9f35b8df4714fd1875b666e57026e8a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:34.251253 14057 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:34.251770 14073 mvcc.cc:205] Tried to move back new op lower bound from 7315451061244096512 to 7315451061034123264. Current Snapshot: MvccSnapshot[applied={T|T < 7315451061244096512}]
I20260806 06:46:34.251912 14058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a9f35b8df4714fd1875b666e57026e8a" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3032a4ae73744fd593e028910ee7ed2f" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43343 } } peers { permanent_uuid: "a9f35b8df4714fd1875b666e57026e8a" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 42589 } attrs { promote: false } } peers { permanent_uuid: "b25ebdad004a49358fe7c1fe808f574f" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 42415 } attrs { promote: false } } }
I20260806 06:46:34.252027 14058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b25ebdad004a49358fe7c1fe808f574f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:46:34.256351 14059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.252933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:37667 (local address 127.0.0.1:53510)
0806 06:46:34.252971 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.252980 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.253017 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.253617 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.253618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.253620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.253649 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.253650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254291 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.254292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.255405 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.255408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256119 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.256181 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256185 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.256241 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.256285 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:46:34.256356 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.253007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.6.57.66:37667)
0806 06:46:34.253192 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.253195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.253215 (+ 20us) server_negotiation.cc:407] Connection header received
0806 06:46:34.253245 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.253246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.253278 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.253621 (+ 343us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.253710 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254191 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.255508 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256039 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256131 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256146 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.256248 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.256288 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
I20260806 06:46:34.256470 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.252984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:46355 (local address 127.0.0.1:43574)
0806 06:46:34.253017 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.253027 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.253070 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.253502 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.253504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.253514 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.253568 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.253571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254615 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.254616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.255540 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.255542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256247 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.256315 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256318 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.256349 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.256389 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":38}
I20260806 06:46:34.256783 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.253088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41819 (local address 127.0.0.1:36978)
0806 06:46:34.253310 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.253319 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.253360 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.253596 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.253597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.253599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.253645 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.253647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254599 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.254599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.255781 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.255784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256538 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.256609 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256613 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.256670 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.256711 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":20,"thread_start_us":59,"threads_started":1}
I20260806 06:46:34.256779 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.253267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.6.57.67:46355)
0806 06:46:34.253372 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.253374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.253382 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:34.253413 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.253413 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.253421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.253498 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.253727 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254480 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.255629 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256151 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256634 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256650 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.256670 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.256712 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260806 06:46:34.256812 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.253167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.6.57.65:41819)
0806 06:46:34.253447 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.253449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.253456 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:34.253481 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.253482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.253492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.253547 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.253711 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.254533 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.255864 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.256415 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.256476 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.256491 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.256696 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.256743 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":84,"threads_started":1}
I20260806 06:46:34.256966 13777 tablet_service.cc:1467] Processing CreateTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 (DEFAULT_TABLE table=test-table [id=5b08f661ad1c41ef9f79de5af1a75073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:34.257033 13636 tablet_service.cc:1467] Processing CreateTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 (DEFAULT_TABLE table=test-table [id=5b08f661ad1c41ef9f79de5af1a75073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:34.257115 13777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ad32b59b384530ad1051ef5fb0b000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:34.257124 13918 tablet_service.cc:1467] Processing CreateTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 (DEFAULT_TABLE table=test-table [id=5b08f661ad1c41ef9f79de5af1a75073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:34.257175 13636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ad32b59b384530ad1051ef5fb0b000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:34.257229 13918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3ad32b59b384530ad1051ef5fb0b000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:34.258656 14082 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Bootstrap starting.
I20260806 06:46:34.258904 14083 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: Bootstrap starting.
I20260806 06:46:34.259064 14082 tablet_bootstrap.cc:654] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:34.259204 14081 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Bootstrap starting.
I20260806 06:46:34.259328 14083 tablet_bootstrap.cc:654] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:34.259598 14082 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: No bootstrap required, opened a new log
I20260806 06:46:34.259677 14081 tablet_bootstrap.cc:654] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:34.259685 14082 ts_tablet_manager.cc:1397] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:34.259820 14082 raft_consensus.cc:348] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.259889 14082 raft_consensus.cc:374] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:34.259909 14082 raft_consensus.cc:729] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bda12ce5b54fc28ddb550965e56938, State: Initialized, Role: FOLLOWER
I20260806 06:46:34.259963 14082 consensus_queue.cc:260] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.260180 14081 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: No bootstrap required, opened a new log
I20260806 06:46:34.260448 14081 ts_tablet_manager.cc:1397] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:34.260563 14083 tablet_bootstrap.cc:492] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: No bootstrap required, opened a new log
I20260806 06:46:34.260658 14083 ts_tablet_manager.cc:1397] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:34.260169 14082 ts_tablet_manager.cc:1428] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:34.260766 13988 heartbeater.cc:503] Master 127.6.57.125:42589 was elected leader, sending a full tablet report...
I20260806 06:46:34.260567 14081 raft_consensus.cc:348] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.260963 14081 raft_consensus.cc:374] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:34.260995 14081 raft_consensus.cc:729] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed661da5b2654ac2afe7e25d3be8237b, State: Initialized, Role: FOLLOWER
I20260806 06:46:34.260926 14083 raft_consensus.cc:348] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.261050 14081 consensus_queue.cc:260] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.261165 14083 raft_consensus.cc:374] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:34.261237 14083 raft_consensus.cc:729] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168ed8a28d2a4214ab7e36d9b6f8a68d, State: Initialized, Role: FOLLOWER
I20260806 06:46:34.261236 14081 ts_tablet_manager.cc:1428] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:34.261355 14083 consensus_queue.cc:260] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.261580 14083 ts_tablet_manager.cc:1428] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:34.261644 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.261424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:57569)
0806 06:46:34.261567 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.261608 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 06:46:34.261924 13848 heartbeater.cc:503] Master 127.6.57.125:42589 was elected leader, sending a full tablet report...
I20260806 06:46:34.262300 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.262093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:58231)
0806 06:46:34.262245 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.262276 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260806 06:46:34.262542 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.260378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:40103)
0806 06:46:34.262446 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.262510 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2044,"thread_start_us":76,"threads_started":1}
I20260806 06:46:34.262661 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.261869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:34449)
0806 06:46:34.262572 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.262602 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":75,"threads_started":1}
I20260806 06:46:34.262785 13707 heartbeater.cc:503] Master 127.6.57.125:42589 was elected leader, sending a full tablet report...
I20260806 06:46:34.262893 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.262802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:35279)
0806 06:46:34.262843 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.262872 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:46:34.263096 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.262979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:53125)
0806 06:46:34.263042 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.263068 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 06:46:34.284171 13989 tablet.cc:2369] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:34.453614 14087 raft_consensus.cc:482] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:34.453711 14087 raft_consensus.cc:504] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.454159 14087 leader_election.cc:288] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667), 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
I20260806 06:46:34.456975 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.454326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:37667 (local address 127.6.57.67:33321)
0806 06:46:34.454381 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.454392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.454446 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.454635 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.454636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.454639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.454722 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.454723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.455371 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.455372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.456160 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.456162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.456719 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.456791 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.456794 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.456866 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.456905 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:46:34.456975 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.454402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:33321 (local address 127.6.57.66:37667)
0806 06:46:34.454449 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.454451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.454479 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:46:34.454534 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.454535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.454548 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.454627 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.454794 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.455285 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456277 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.456628 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456716 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.456728 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.456875 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.456913 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 06:46:34.457105 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.454417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:47207 (local address 127.6.57.65:41819)
0806 06:46:34.454510 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.454513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.454668 (+ 155us) server_negotiation.cc:407] Connection header received
0806 06:46:34.454770 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.454772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.454786 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.454856 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:34.454998 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.455460 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456384 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.456730 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456810 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.456822 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.456987 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.457021 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:34.457211 13798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ad32b59b384530ad1051ef5fb0b000" candidate_uuid: "25bda12ce5b54fc28ddb550965e56938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed661da5b2654ac2afe7e25d3be8237b" is_pre_election: true
I20260806 06:46:34.457283 14093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.454356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41819 (local address 127.6.57.67:47207)
0806 06:46:34.454594 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.454602 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:34.454683 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:34.454861 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:34.454862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:34.454863 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:34.454928 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.454929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.455557 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.455558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.456286 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:34.456287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.456829 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.456830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:34.456900 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.456903 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:34.456976 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:46:34.457013 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":139,"threads_started":1}
I20260806 06:46:34.457350 13798 raft_consensus.cc:2454] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25bda12ce5b54fc28ddb550965e56938 in term 0.
I20260806 06:46:34.457509 13871 leader_election.cc:302] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bda12ce5b54fc28ddb550965e56938, ed661da5b2654ac2afe7e25d3be8237b; no voters:
I20260806 06:46:34.457554 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ad32b59b384530ad1051ef5fb0b000" candidate_uuid: "25bda12ce5b54fc28ddb550965e56938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" is_pre_election: true
I20260806 06:46:34.457650 14087 raft_consensus.cc:2781] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:34.457659 13656 raft_consensus.cc:2454] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25bda12ce5b54fc28ddb550965e56938 in term 0.
I20260806 06:46:34.457695 14087 raft_consensus.cc:482] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:34.457719 14087 raft_consensus.cc:3037] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:34.458410 14087 raft_consensus.cc:504] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.458576 14087 leader_election.cc:288] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [CANDIDATE]: Term 1 election: Requested vote from peers ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667), 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819)
I20260806 06:46:34.458767 13798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ad32b59b384530ad1051ef5fb0b000" candidate_uuid: "25bda12ce5b54fc28ddb550965e56938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed661da5b2654ac2afe7e25d3be8237b"
I20260806 06:46:34.458851 13798 raft_consensus.cc:3037] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:34.458732 13656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3ad32b59b384530ad1051ef5fb0b000" candidate_uuid: "25bda12ce5b54fc28ddb550965e56938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d"
I20260806 06:46:34.458896 13656 raft_consensus.cc:3037] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:34.459534 13798 raft_consensus.cc:2454] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25bda12ce5b54fc28ddb550965e56938 in term 1.
I20260806 06:46:34.459539 13656 raft_consensus.cc:2454] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25bda12ce5b54fc28ddb550965e56938 in term 1.
I20260806 06:46:34.459707 13871 leader_election.cc:302] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bda12ce5b54fc28ddb550965e56938, ed661da5b2654ac2afe7e25d3be8237b; no voters:
I20260806 06:46:34.459803 14087 raft_consensus.cc:2781] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:34.459950 14087 raft_consensus.cc:686] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 1 LEADER]: Becoming Leader. State: Replica: 25bda12ce5b54fc28ddb550965e56938, State: Running, Role: LEADER
I20260806 06:46:34.460007 14087 consensus_queue.cc:237] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } }
I20260806 06:46:34.460317 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.460230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:59813)
0806 06:46:34.460261 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.460298 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 06:46:34.460887 13326 catalog_manager.cc:5495] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "25bda12ce5b54fc28ddb550965e56938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25bda12ce5b54fc28ddb550965e56938" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 46355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:34.481949 14080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.479880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43588 (local address 127.6.57.67:46355)
0806 06:46:34.479943 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:34.479945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:34.479953 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:34.479984 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:34.479985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:34.480003 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:34.480061 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:34.480181 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.480630 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.481360 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:34.481531 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:34.481579 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:34.481700 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:34.481794 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:34.481842 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:34.481881 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 06:46:34.481896 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:34.483179 13656 raft_consensus.cc:1261] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d [term 1 FOLLOWER]: Refusing update from remote peer 25bda12ce5b54fc28ddb550965e56938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:34.483273 13798 raft_consensus.cc:1261] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Refusing update from remote peer 25bda12ce5b54fc28ddb550965e56938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:34.483470 14094 consensus_queue.cc:1035] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "168ed8a28d2a4214ab7e36d9b6f8a68d" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:34.483619 14087 consensus_queue.cc:1035] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed661da5b2654ac2afe7e25d3be8237b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 37667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:34.484550 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.484427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:41587)
0806 06:46:34.484472 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.484516 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:46:34.484709 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.484602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:51389)
0806 06:46:34.484647 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.484675 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 06:46:34.486080 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.485988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.66:34959)
0806 06:46:34.486025 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.486049 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 06:46:34.486225 14092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.486130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.65:43115)
0806 06:46:34.486167 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.486192 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:46:34.486372 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.486267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:42553)
0806 06:46:34.486323 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.486345 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:46:34.486505 14100 mvcc.cc:205] Tried to move back new op lower bound from 7315451062200594432 to 7315451062108880896. Current Snapshot: MvccSnapshot[applied={T|T < 7315451062200594432}]
I20260806 06:46:34.486871 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.486766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43343 (local address 127.6.57.67:59783)
0806 06:46:34.486799 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.486844 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:43343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:46:34.488785 13326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:34.488879 13326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:34.490175 13326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9f35b8df4714fd1875b666e57026e8a: Sending DeleteTablet for 3 replicas of tablet d3ad32b59b384530ad1051ef5fb0b000
I20260806 06:46:34.490463 13918 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:34 UTC) from {username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:43574
I20260806 06:46:34.490523 13636 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:34 UTC) from {username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:36978
I20260806 06:46:34.490520 13777 tablet_service.cc:1514] Processing DeleteTablet for tablet d3ad32b59b384530ad1051ef5fb0b000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:34 UTC) from {username='oryx', principal='oryx/127.6.57.125@KRBTEST.COM'} at 127.0.0.1:53510
I20260806 06:46:34.490721 14109 tablet_replica.cc:330] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: stopping tablet replica
I20260806 06:46:34.490777 14109 raft_consensus.cc:2229] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:34.490859 14109 raft_consensus.cc:2258] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:34.491014 14110 tablet_replica.cc:330] T d3ad32b59b384530ad1051ef5fb0b000 P 168ed8a28d2a4214ab7e36d9b6f8a68d: stopping tablet replica
I20260806 06:46:34.491099 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13479
I20260806 06:46:34.493603 14109 ts_tablet_manager.cc:1905] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:34.495222 14109 ts_tablet_manager.cc:1918] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:34.495306 14109 log.cc:1196] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/d3ad32b59b384530ad1051ef5fb0b000
I20260806 06:46:34.495555 14109 ts_tablet_manager.cc:1939] T d3ad32b59b384530ad1051ef5fb0b000 P 25bda12ce5b54fc28ddb550965e56938: Deleting consensus metadata
I20260806 06:46:34.495862 13311 catalog_manager.cc:4841] TS 25bda12ce5b54fc28ddb550965e56938 (127.6.57.67:46355): tablet d3ad32b59b384530ad1051ef5fb0b000 (table test-table [id=5b08f661ad1c41ef9f79de5af1a75073]) successfully deleted
I20260806 06:46:34.498605 14111 tablet_replica.cc:330] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: stopping tablet replica
I20260806 06:46:34.498679 14111 raft_consensus.cc:2229] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:34.498729 14111 raft_consensus.cc:2258] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:34.499027 14111 ts_tablet_manager.cc:1905] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:34.500330 14111 ts_tablet_manager.cc:1918] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:34.500397 14111 log.cc:1196] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/d3ad32b59b384530ad1051ef5fb0b000
I20260806 06:46:34.500713 14111 ts_tablet_manager.cc:1939] T d3ad32b59b384530ad1051ef5fb0b000 P ed661da5b2654ac2afe7e25d3be8237b: Deleting consensus metadata
I20260806 06:46:34.501026 13310 catalog_manager.cc:4841] TS ed661da5b2654ac2afe7e25d3be8237b (127.6.57.66:37667): tablet d3ad32b59b384530ad1051ef5fb0b000 (table test-table [id=5b08f661ad1c41ef9f79de5af1a75073]) successfully deleted
I20260806 06:46:34.502904 14078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:34.502802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41819 (local address 127.0.0.1:36984)
0806 06:46:34.502839 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:34.502866 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:41819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:46:34.503064 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13714
W20260806 06:46:34.503082 13311 catalog_manager.cc:4584] TS 168ed8a28d2a4214ab7e36d9b6f8a68d (127.6.57.65:41819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3ad32b59b384530ad1051ef5fb0b000: Network error: Client connection negotiation failed: client connection to 127.6.57.65:41819: connect: Connection refused (error 111)
I20260806 06:46:34.516033 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13854
I20260806 06:46:34.522300 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13189
I20260806 06:46:34.527561 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 13382
2026-08-06T06:46:34Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6134 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:34 dist-test-slave-5l3k krb5kdc[14114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:34 dist-test-slave-5l3k krb5kdc[14114](info): set up 2 sockets
Aug 06 06:46:34 dist-test-slave-5l3k krb5kdc[14114](info): commencing operation
krb5kdc: starting...
W20260806 06:46:35.629031 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:35 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab.
Entry for principal oryx/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab.
I20260806 06:46:35.753865 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42585
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab
--principal=oryx/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:35.835695 14128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:35.835899 14128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:35.835922 14128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:35.837033 14128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:35.837066 14128 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 06:46:35.837080 14128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:35.837092 14128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:35.837105 14128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:35.837117 14128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:35.838692 14128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42585
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.126.keytab
--principal=oryx/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:35.838896 14128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:35.839068 14128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:35.843825 14133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:35.843803 14136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:35.843899 14128 server_base.cc:1034] running on GCE node
W20260806 06:46:35.843822 14134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:35.844372 14128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:35.844676 14128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:35.845844 14128 hybrid_clock.cc:648] HybridClock initialized: now 1785998795845836 us; error 35 us; skew 500 ppm
Aug 06 06:46:35 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998795, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:35.848345 14128 init.cc:377] Logged in from keytab as oryx/127.6.57.126@KRBTEST.COM (short username oryx)
I20260806 06:46:35.848801 14128 webserver.cc:460] Webserver started at http://127.6.57.126:46301/ using document root <none> and password file <none>
I20260806 06:46:35.849047 14128 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:35.849113 14128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:35.849274 14128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:35.850332 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "f642285ac1234dfb8da611b54cd7cdf2"
format_stamp: "Formatted at 2026-08-06 06:46:35 on dist-test-slave-5l3k"
server_key: "97b0f2ba1366bce485c6348e4ab1d0d5"
server_key_iv: "cc8e0a5767bcaacde1a9474ffa1900a0"
server_key_version: "encryptionkey@0"
I20260806 06:46:35.850719 14128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "f642285ac1234dfb8da611b54cd7cdf2"
format_stamp: "Formatted at 2026-08-06 06:46:35 on dist-test-slave-5l3k"
server_key: "97b0f2ba1366bce485c6348e4ab1d0d5"
server_key_iv: "cc8e0a5767bcaacde1a9474ffa1900a0"
server_key_version: "encryptionkey@0"
I20260806 06:46:35.852041 14128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:35.852878 14143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:35.853085 14128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:35.853190 14128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "f642285ac1234dfb8da611b54cd7cdf2"
format_stamp: "Formatted at 2026-08-06 06:46:35 on dist-test-slave-5l3k"
server_key: "97b0f2ba1366bce485c6348e4ab1d0d5"
server_key_iv: "cc8e0a5767bcaacde1a9474ffa1900a0"
server_key_version: "encryptionkey@0"
I20260806 06:46:35.853274 14128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:35.864454 14128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:35.866281 14128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:35.866441 14128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:35.871570 14128 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:42585
I20260806 06:46:35.871621 14209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:42585 every 8 connection(s)
I20260806 06:46:35.871937 14128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:35.872682 14210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:35.875360 14210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Bootstrap starting.
I20260806 06:46:35.876086 14210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:35.876427 14210 log.cc:824] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:35.877166 14210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: No bootstrap required, opened a new log
I20260806 06:46:35.878710 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14128
I20260806 06:46:35.878789 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:35.879004 6373 external_mini_cluster.cc:1428] Setting key bd9ad890394c96ceafec1ea4609bfaff
I20260806 06:46:35.879742 14210 raft_consensus.cc:348] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } }
I20260806 06:46:35.879925 14210 raft_consensus.cc:374] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:35.879979 14210 raft_consensus.cc:729] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f642285ac1234dfb8da611b54cd7cdf2, State: Initialized, Role: FOLLOWER
I20260806 06:46:35.880158 14210 consensus_queue.cc:260] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } }
I20260806 06:46:35.880267 14210 raft_consensus.cc:388] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:35.880335 14210 raft_consensus.cc:482] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:35.880405 14210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:35.880971 14210 raft_consensus.cc:504] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } }
I20260806 06:46:35.881070 14210 leader_election.cc:302] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f642285ac1234dfb8da611b54cd7cdf2; no voters:
I20260806 06:46:35.881270 14210 leader_election.cc:288] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:35.881309 14215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:35.881467 14215 raft_consensus.cc:686] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [term 1 LEADER]: Becoming Leader. State: Replica: f642285ac1234dfb8da611b54cd7cdf2, State: Running, Role: LEADER
I20260806 06:46:35.881557 14210 sys_catalog.cc:564] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:35.881614 14215 consensus_queue.cc:237] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } }
I20260806 06:46:35.881943 14215 sys_catalog.cc:455] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f642285ac1234dfb8da611b54cd7cdf2. Latest consensus state: current_term: 1 leader_uuid: "f642285ac1234dfb8da611b54cd7cdf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } } }
I20260806 06:46:35.882057 14215 sys_catalog.cc:458] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:35.881953 14217 sys_catalog.cc:455] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f642285ac1234dfb8da611b54cd7cdf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f642285ac1234dfb8da611b54cd7cdf2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42585 } } }
I20260806 06:46:35.882192 14217 sys_catalog.cc:458] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:35.883085 14221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:35.883610 14221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:35.885140 14221 catalog_manager.cc:1347] Generated new cluster ID: 88065d79b2224f1aa5d695e9b0225138
I20260806 06:46:35.885217 14221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:35 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:35.891685 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:35.879584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.6.57.126:42585)
0806 06:46:35.879817 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:35.879830 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:35.879856 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:46:35.879973 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:35.879981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:35.880076 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:35.880222 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:35.880945 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:35.882205 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:35.884156 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:35.884583 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:35.884697 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:35.887899 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:35.887904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:35.887904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:35.887915 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:35.889953 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:35.890404 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:35.890407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:35.890408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:35.890554 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:35.890795 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:35.890796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:35.890796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:35.891067 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:35.891205 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:35.891343 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:46:35.891467 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":117,"threads_started":1}
I20260806 06:46:35.893843 14221 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:35.894693 14221 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:35.899706 14221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Generated new TSK 0
I20260806 06:46:35.899890 14221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
I20260806 06:46:35.927248 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:36.010836 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:36.011023 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:36.011053 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:36.012231 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:36.012274 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:36.012293 14236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:36.012310 14236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:36.013818 14236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:36.014073 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:36.014312 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:36.019378 14241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:36.019418 14242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:36.019477 14236 server_base.cc:1034] running on GCE node
W20260806 06:46:36.019382 14244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:36.019762 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:36.019992 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:36.021162 14236 hybrid_clock.cc:648] HybridClock initialized: now 1785998796021140 us; error 34 us; skew 500 ppm
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:36.036757 14236 init.cc:377] Logged in from keytab as oryx/127.6.57.65@KRBTEST.COM (short username oryx)
I20260806 06:46:36.037392 14236 webserver.cc:460] Webserver started at http://127.6.57.65:39403/ using document root <none> and password file <none>
I20260806 06:46:36.037621 14236 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:36.037668 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:36.037773 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:36.038924 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "b005b14441fe370e20ec4d36061d4c77"
server_key_iv: "1173dd2b446b7461f0825ae23e153b7d"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.039242 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "b005b14441fe370e20ec4d36061d4c77"
server_key_iv: "1173dd2b446b7461f0825ae23e153b7d"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.040583 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:36.041278 14251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.041451 14236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:36.041556 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "b005b14441fe370e20ec4d36061d4c77"
server_key_iv: "1173dd2b446b7461f0825ae23e153b7d"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.041635 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:36.059377 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:36.061746 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:36.061915 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:36.062600 14236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:36.062660 14236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.062702 14236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:36.062729 14236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.070077 14236 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:44889
I20260806 06:46:36.070180 14370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:44889 every 8 connection(s)
I20260806 06:46:36.070528 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:36.072281 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14236
I20260806 06:46:36.072410 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:36.072657 6373 external_mini_cluster.cc:1428] Setting key 9a2f9b6e6bd41d240ac6671c2c37665d
I20260806 06:46:36.073028 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.063017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:43681 (local address 127.6.57.126:42585)
0806 06:46:36.063088 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.063090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.063329 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:46:36.063833 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.063834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.063851 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.063933 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.064802 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.065660 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.067116 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.067480 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.067575 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.070068 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.070071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.070071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.070080 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.071649 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.072009 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.072010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.072011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.072060 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.072330 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.072331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.072332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.072486 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.072569 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.072824 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.072925 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:46:36.073181 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.063013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.65:43681)
0806 06:46:36.063191 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.063232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.063735 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.064027 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.064032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.064403 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.064689 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.064697 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.065772 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.065774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.066984 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.066988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.067605 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.067607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.067732 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.068212 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.068219 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.069968 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.071722 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.071728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.071730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.071944 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.072172 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.072173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.072173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.072266 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.072557 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.072563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.072671 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.072795 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.072950 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260806 06:46:36.073608 14371 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42585
I20260806 06:46:36.073690 14371 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:36.073882 14371 heartbeater.cc:511] Master 127.6.57.126:42585 requested a full tablet report, sending...
I20260806 06:46:36.074429 14161 ts_manager.cc:194] Registered new tserver with Master: 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889)
I20260806 06:46:36.075371 14161 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:43681
I20260806 06:46:36.080884 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.074856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:38543 (local address 127.6.57.126:42585)
0806 06:46:36.074890 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.074892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.074899 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:36.074919 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.074920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.074930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.074989 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.075345 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.076007 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.077051 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.077220 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.077272 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.078269 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.078270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.078270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.078276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.079694 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.080111 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.080112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.080112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.080160 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.080423 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.080424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.080424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.080531 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.080588 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.080770 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.080829 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:46:36.080986 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.074639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.65:38543)
0806 06:46:36.074789 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.074800 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.074851 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.075001 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.075002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.075199 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.075264 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.075266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.076116 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.076117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.076938 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.076942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.077294 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.077295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.077363 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.077745 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.077750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.078172 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.079779 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.079780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.079781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.080022 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.080242 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.080243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.080244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.080328 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.080599 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.080600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.080657 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.080745 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.080851 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
I20260806 06:46:36.099427 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:36.179665 14380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:36.179888 14380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:36.179929 14380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:36.181097 14380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:36.181155 14380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:36.181190 14380 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:36.181221 14380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:36.182816 14380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:36.183073 14380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:36.183293 14380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:36.188134 14388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:36.188150 14386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:36.188133 14385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:36.188313 14380 server_base.cc:1034] running on GCE node
I20260806 06:46:36.188519 14380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:36.188757 14380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:36.189919 14380 hybrid_clock.cc:648] HybridClock initialized: now 1785998796189924 us; error 46 us; skew 500 ppm
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:36.192852 14380 init.cc:377] Logged in from keytab as oryx/127.6.57.66@KRBTEST.COM (short username oryx)
I20260806 06:46:36.193334 14380 webserver.cc:460] Webserver started at http://127.6.57.66:45737/ using document root <none> and password file <none>
I20260806 06:46:36.193602 14380 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:36.193677 14380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:36.193810 14380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:36.194875 14380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "a55a1b06905443e2910f7494d17c90aa"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "8760fdb5fdcc221152a1bb5b950534d2"
server_key_iv: "c8dbc863135cf25c202872f4cbc91701"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.195250 14380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "a55a1b06905443e2910f7494d17c90aa"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "8760fdb5fdcc221152a1bb5b950534d2"
server_key_iv: "c8dbc863135cf25c202872f4cbc91701"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.196552 14380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:46:36.197279 14395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.197444 14380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.197553 14380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "a55a1b06905443e2910f7494d17c90aa"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "8760fdb5fdcc221152a1bb5b950534d2"
server_key_iv: "c8dbc863135cf25c202872f4cbc91701"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.197631 14380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:36.246654 14380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:36.248870 14380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:36.248997 14380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:36.249542 14380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:36.249622 14380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.249660 14380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:36.249686 14380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.257210 14380 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:46429
I20260806 06:46:36.257293 14515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:46429 every 8 connection(s)
I20260806 06:46:36.257637 14380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:36.260118 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.250189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:45629 (local address 127.6.57.126:42585)
0806 06:46:36.250243 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.250246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.250469 (+ 223us) server_negotiation.cc:407] Connection header received
0806 06:46:36.251120 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.251121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.251145 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.251228 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.252144 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.252970 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.254334 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.254709 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.254805 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.257276 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.257278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.257278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.257286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.258893 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.259360 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.259360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.259361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.259412 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.259636 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.259636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.259637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.259734 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.259793 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.259995 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.260052 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:36.260385 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.250174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.66:45629)
0806 06:46:36.250346 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.250386 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.250994 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.251360 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.251365 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.251734 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.252048 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.252055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.253071 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.253073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.254076 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.254079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.254821 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.254823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.254969 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.255458 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.255467 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.257168 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.259057 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.259062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.259064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.259288 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.259478 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.259479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.259479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.259575 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.259795 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.259798 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.259903 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.259982 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.260116 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260806 06:46:36.260900 14516 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42585
I20260806 06:46:36.260982 14516 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:36.261183 14516 heartbeater.cc:511] Master 127.6.57.126:42585 requested a full tablet report, sending...
I20260806 06:46:36.261647 14161 ts_manager.cc:194] Registered new tserver with Master: a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429)
I20260806 06:46:36.262153 14161 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:45629
I20260806 06:46:36.264564 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14380
I20260806 06:46:36.264696 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:36.264928 6373 external_mini_cluster.cc:1428] Setting key ad4ad79fd7e6083b788b9171bf2f1ef8
I20260806 06:46:36.267957 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.262045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:58915 (local address 127.6.57.126:42585)
0806 06:46:36.262087 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.262089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.262238 (+ 149us) server_negotiation.cc:407] Connection header received
0806 06:46:36.262264 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.262265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.262277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.262326 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.262736 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.263332 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.263953 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.264129 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.264184 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.265183 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.265184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.265185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.265191 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.266854 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.267167 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.267167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.267168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.267227 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.267400 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.267401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.267401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.267610 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.267679 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.267836 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.267904 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:36.268090 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.261989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.66:58915)
0806 06:46:36.262149 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.262157 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.262207 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.262332 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.262334 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.262551 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.262638 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.262640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.263427 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.263428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.263888 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.263889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.264208 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.264208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.264251 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.264672 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.264677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.265077 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.266935 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.266936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.266937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.267095 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.267292 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.267293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.267293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.267344 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.267685 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.267686 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.267737 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.267819 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.267938 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
I20260806 06:46:36.289449 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:42585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:36.370910 14526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:36.371093 14526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:36.371115 14526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:36.372189 14526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:36.372226 14526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:36.372246 14526 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 06:46:36.372262 14526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:36.373638 14526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.57.126:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:36.373853 14526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:36.374076 14526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:36.379117 14532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:36.379056 14531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:36.379077 14534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:36.379323 14526 server_base.cc:1034] running on GCE node
I20260806 06:46:36.379547 14526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:36.379777 14526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:36.380915 14526 hybrid_clock.cc:648] HybridClock initialized: now 1785998796380905 us; error 35 us; skew 500 ppm
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:36.383718 14526 init.cc:377] Logged in from keytab as oryx/127.6.57.67@KRBTEST.COM (short username oryx)
I20260806 06:46:36.384181 14526 webserver.cc:460] Webserver started at http://127.6.57.67:37291/ using document root <none> and password file <none>
I20260806 06:46:36.384436 14526 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:36.384485 14526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:36.384668 14526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:36.385761 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "e791a9d28aa24be4afc48c95d82e19e3"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "f8f99fe94d1d1ef868b21a5044e558a0"
server_key_iv: "db74e1eff9792afda9015a0b987bdd9f"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.386142 14526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "e791a9d28aa24be4afc48c95d82e19e3"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "f8f99fe94d1d1ef868b21a5044e558a0"
server_key_iv: "db74e1eff9792afda9015a0b987bdd9f"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.387557 14526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:36.388314 14541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.388494 14526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.388592 14526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "e791a9d28aa24be4afc48c95d82e19e3"
format_stamp: "Formatted at 2026-08-06 06:46:36 on dist-test-slave-5l3k"
server_key: "f8f99fe94d1d1ef868b21a5044e558a0"
server_key_iv: "db74e1eff9792afda9015a0b987bdd9f"
server_key_version: "encryptionkey@0"
I20260806 06:46:36.388664 14526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:36.412114 14526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:36.413899 14526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:36.414077 14526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:36.414602 14526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:36.414645 14526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.414667 14526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:36.414681 14526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.422365 14526 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:34911
I20260806 06:46:36.422434 14661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:34911 every 8 connection(s)
I20260806 06:46:36.422777 14526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:36.424263 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14526
I20260806 06:46:36.424365 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:36.424576 6373 external_mini_cluster.cc:1428] Setting key d2d3b5c3673734d24298307a6ecf728a
I20260806 06:46:36.425606 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.415043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:55355 (local address 127.6.57.126:42585)
0806 06:46:36.415110 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.415114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.415302 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:46:36.415811 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.415812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.415826 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.415887 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.416629 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.417544 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.419233 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.419613 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.419713 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.422513 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.422514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.422515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.422522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.424008 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.424466 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.424467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.424468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.424535 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.424858 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.424859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.424860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.425004 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.425069 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.425429 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.425511 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:36.425784 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.415013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.67:55355)
0806 06:46:36.415169 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.415208 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.415703 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.415954 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.415958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.416232 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.416497 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.416509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.417650 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.417651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.419134 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.419138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.419749 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.419750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.419897 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.420386 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.420398 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.422386 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.424129 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.424134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.424136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.424391 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.424605 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.424606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.424606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.424763 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.425079 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.425085 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.425263 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.425385 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.425576 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260806 06:46:36.426220 14662 heartbeater.cc:347] Connected to a master server at 127.6.57.126:42585
I20260806 06:46:36.426306 14662 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:36.426541 14662 heartbeater.cc:511] Master 127.6.57.126:42585 requested a full tablet report, sending...
I20260806 06:46:36.426949 14162 ts_manager.cc:194] Registered new tserver with Master: e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911)
I20260806 06:46:36.427441 14162 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:55355
I20260806 06:46:36.432181 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.427402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:43447 (local address 127.6.57.126:42585)
0806 06:46:36.427571 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.427572 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.427579 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:36.427594 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.427595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.427603 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.427681 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.428049 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.428540 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.429167 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.429343 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.429392 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.430188 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.430189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.430189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.430194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.431380 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.431670 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.431671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.431671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.431719 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.431859 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.431859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.431860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.431936 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.431979 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.432079 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.432129 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260806 06:46:36.432202 14668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.427318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.6.57.67:43447)
0806 06:46:36.427457 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.427468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.427518 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.427686 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.427687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.427898 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.427973 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.427976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.428625 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.428626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.429090 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.429092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.429403 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.429404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.429450 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.429845 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.429850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.430120 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.431451 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.431451 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.431452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.431604 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.431774 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.431774 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.431775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.431823 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.431983 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.431983 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.432016 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.432063 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.432139 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260806 06:46:36.436244 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:46:36.458674 14214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:36.453229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.6.57.126:42585)
0806 06:46:36.453309 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.453311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.453323 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:36.453352 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.453353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.453363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.453415 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.453931 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.454442 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.455237 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.455482 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.455543 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.458566 (+ 3023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34796: BlockingRecv error: recv got EOF from 127.0.0.1:34796 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.466231 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.459478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.6.57.126:42585)
0806 06:46:36.459548 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.459551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.459633 (+ 82us) server_negotiation.cc:407] Connection header received
0806 06:46:36.459704 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.459706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.459715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.459761 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.460257 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.460732 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.461584 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.461759 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.461820 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.464163 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.464164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.464164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.464171 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.465356 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.465652 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.465653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.465653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.465704 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.465862 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.465862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.465862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.465955 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.466031 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.466111 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.466170 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:46:36.467986 14162 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:36.469488 14162 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:36.476181 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.472206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.6.57.67:34911)
0806 06:46:36.472398 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.472402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.472430 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:46:36.472466 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.472468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.472482 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.472637 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.472835 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473875 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475195 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.475731 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475834 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.475923 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.476034 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.476082 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260806 06:46:36.476174 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.471814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:34911 (local address 127.0.0.1:43978)
0806 06:46:36.472316 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.472324 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.472357 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.472703 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.472704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.472706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.472758 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.472759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473950 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.473951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.475124 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.475126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.475838 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.475922 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.475938 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.476015 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.476075 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":27,"thread_start_us":93,"threads_started":1}
I20260806 06:46:36.476349 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.472004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.6.57.66:46429)
0806 06:46:36.472194 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.472200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.472212 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:36.472303 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.472306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.472323 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.472439 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.472704 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473844 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475031 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.475802 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475882 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.475962 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.476199 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.476245 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260806 06:46:36.476392 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.472067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.6.57.65:44889)
0806 06:46:36.472293 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.472297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.472504 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:46:36.472585 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.472587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.472603 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.472675 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.472885 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473625 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475085 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.475825 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.475886 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.475971 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.476237 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.476284 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":80,"threads_started":1}
I20260806 06:46:36.476632 14686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.471808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46429 (local address 127.0.0.1:50624)
0806 06:46:36.472069 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.472131 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.472202 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.472520 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.472524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.472527 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.472602 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.472605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473945 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.473945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.474939 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.474941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.476003 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.476005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.476081 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.476086 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.476491 (+ 405us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.476535 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":32,"thread_start_us":59,"threads_started":1}
I20260806 06:46:36.476773 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.471838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44889 (local address 127.0.0.1:39406)
0806 06:46:36.472436 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.472443 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.472504 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.472751 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.472752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.472754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.472802 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.472803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.473756 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.473757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.474997 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.475000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.476004 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.476005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.476084 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.476086 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.476139 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.476192 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":4,"thread_start_us":74,"threads_started":1}
I20260806 06:46:36.477316 14593 tablet_service.cc:1467] Processing CreateTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 (DEFAULT_TABLE table=test-table [id=b33426ceb91340e49d68a515e441d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.477454 14304 tablet_service.cc:1467] Processing CreateTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 (DEFAULT_TABLE table=test-table [id=b33426ceb91340e49d68a515e441d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.477761 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54997d9a2f184b2bb9175ffade9a08d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.477866 14304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54997d9a2f184b2bb9175ffade9a08d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.479157 14449 tablet_service.cc:1467] Processing CreateTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 (DEFAULT_TABLE table=test-table [id=b33426ceb91340e49d68a515e441d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.479516 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54997d9a2f184b2bb9175ffade9a08d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.480960 14690 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Bootstrap starting.
I20260806 06:46:36.481696 14691 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Bootstrap starting.
I20260806 06:46:36.481875 14690 tablet_bootstrap.cc:654] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.482260 14690 log.cc:824] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:36.482400 14691 tablet_bootstrap.cc:654] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.482785 14691 log.cc:824] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:36.483616 14691 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: No bootstrap required, opened a new log
I20260806 06:46:36.483672 14691 ts_tablet_manager.cc:1397] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:46:36.483809 14694 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Bootstrap starting.
I20260806 06:46:36.484352 14694 tablet_bootstrap.cc:654] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.484359 14690 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: No bootstrap required, opened a new log
I20260806 06:46:36.484434 14690 ts_tablet_manager.cc:1397] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260806 06:46:36.484656 14694 log.cc:824] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:36.485328 14694 tablet_bootstrap.cc:492] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: No bootstrap required, opened a new log
I20260806 06:46:36.485407 14694 ts_tablet_manager.cc:1397] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:36.485968 14691 raft_consensus.cc:348] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.486155 14691 raft_consensus.cc:374] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.486191 14691 raft_consensus.cc:729] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.486332 14691 consensus_queue.cc:260] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.486388 14690 raft_consensus.cc:348] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.486577 14690 raft_consensus.cc:374] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.486637 14690 raft_consensus.cc:729] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.486668 14691 ts_tablet_manager.cc:1428] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:46:36.486797 14371 heartbeater.cc:503] Master 127.6.57.126:42585 was elected leader, sending a full tablet report...
I20260806 06:46:36.486784 14690 consensus_queue.cc:260] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.487064 14690 ts_tablet_manager.cc:1428] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:46:36.487135 14662 heartbeater.cc:503] Master 127.6.57.126:42585 was elected leader, sending a full tablet report...
I20260806 06:46:36.487928 14694 raft_consensus.cc:348] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.488124 14694 raft_consensus.cc:374] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.488168 14694 raft_consensus.cc:729] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55a1b06905443e2910f7494d17c90aa, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.488299 14694 consensus_queue.cc:260] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.488570 14694 ts_tablet_manager.cc:1428] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:36.488627 14516 heartbeater.cc:503] Master 127.6.57.126:42585 was elected leader, sending a full tablet report...
W20260806 06:46:36.508226 14518 tablet.cc:2369] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:36.571295 14372 tablet.cc:2369] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:36.601544 14697 raft_consensus.cc:482] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:36.601662 14697 raft_consensus.cc:504] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.601996 14697 leader_election.cc:288] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889)
I20260806 06:46:36.605984 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.602334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:39033 (local address 127.6.57.65:44889)
0806 06:46:36.602377 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.602379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.602405 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:46:36.602441 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.602441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.602451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.602560 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.602737 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.603493 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605071 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.605622 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605690 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.605700 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.605877 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.605914 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:36.605984 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.602315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:42167 (local address 127.6.57.66:46429)
0806 06:46:36.602363 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.602366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.602522 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:46:36.602594 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.602595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.602609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.602657 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.602823 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.603507 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605091 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.605614 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605690 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.605700 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.605876 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.605914 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:46:36.606025 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.602233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44889 (local address 127.6.57.67:39033)
0806 06:46:36.602281 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.602293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.602344 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.602565 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.602566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.602568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.602655 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.602657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.603609 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.603610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.604962 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.604966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.605707 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.605797 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.605799 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.605857 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.605908 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":97}
I20260806 06:46:36.606163 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.602237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46429 (local address 127.6.57.67:42167)
0806 06:46:36.602430 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.602438 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.602530 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.602684 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.602685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.602686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.602755 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.602756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.603609 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.603610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.604962 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.604966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.605708 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.605708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.605795 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.605799 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.605857 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.605908 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":82,"thread_start_us":94,"threads_started":1}
I20260806 06:46:36.606514 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54997d9a2f184b2bb9175ffade9a08d1" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" is_pre_election: true
I20260806 06:46:36.606622 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54997d9a2f184b2bb9175ffade9a08d1" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa" is_pre_election: true
I20260806 06:46:36.606734 14324 raft_consensus.cc:2454] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 0.
I20260806 06:46:36.606802 14469 raft_consensus.cc:2454] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 0.
I20260806 06:46:36.607028 14544 leader_election.cc:302] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:36.607187 14697 raft_consensus.cc:2781] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:36.607244 14697 raft_consensus.cc:482] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:36.607273 14697 raft_consensus.cc:3037] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.608222 14697 raft_consensus.cc:504] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.608371 14697 leader_election.cc:288] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 election: Requested vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889)
I20260806 06:46:36.608537 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54997d9a2f184b2bb9175ffade9a08d1" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf"
I20260806 06:46:36.608557 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54997d9a2f184b2bb9175ffade9a08d1" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa"
I20260806 06:46:36.608629 14324 raft_consensus.cc:3037] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.608644 14469 raft_consensus.cc:3037] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.609323 14469 raft_consensus.cc:2454] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 1.
I20260806 06:46:36.609402 14324 raft_consensus.cc:2454] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 1.
I20260806 06:46:36.609566 14544 leader_election.cc:302] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:36.609688 14697 raft_consensus.cc:2781] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:36.609902 14697 raft_consensus.cc:686] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 LEADER]: Becoming Leader. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Running, Role: LEADER
I20260806 06:46:36.610011 14697 consensus_queue.cc:237] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } }
I20260806 06:46:36.610970 14160 catalog_manager.cc:5495] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "e791a9d28aa24be4afc48c95d82e19e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:36.623958 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.621874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.6.57.67:34911)
0806 06:46:36.621934 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.621936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.621950 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:46:36.621976 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.621976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.621991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.622057 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:36.622182 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.622657 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.623327 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.623492 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.623544 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.623716 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.623803 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.623860 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.623897 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.623912 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:36.625554 14324 raft_consensus.cc:1261] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Refusing update from remote peer e791a9d28aa24be4afc48c95d82e19e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.625569 14469 raft_consensus.cc:1261] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Refusing update from remote peer e791a9d28aa24be4afc48c95d82e19e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.625877 14700 consensus_queue.cc:1035] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.626016 14697 consensus_queue.cc:1035] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.629004 14705 mvcc.cc:205] Tried to move back new op lower bound from 7315451070975836160 to 7315451070915395584. Current Snapshot: MvccSnapshot[applied={T|T < 7315451070975836160}]
I20260806 06:46:36.629757 14704 mvcc.cc:205] Tried to move back new op lower bound from 7315451070975836160 to 7315451070915395584. Current Snapshot: MvccSnapshot[applied={T|T < 7315451070975836160 or (T in {7315451070975836160})}]
I20260806 06:46:36.631108 14160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:36.631189 14160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:36.631856 14160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Sending DeleteTablet for 3 replicas of tablet 54997d9a2f184b2bb9175ffade9a08d1
I20260806 06:46:36.632151 14593 tablet_service.cc:1514] Processing DeleteTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43978
I20260806 06:46:36.632272 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 06:46:36.632309 14160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:36.632480 14160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:36.632608 14304 tablet_service.cc:1514] Processing DeleteTablet for tablet 54997d9a2f184b2bb9175ffade9a08d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:39406
I20260806 06:46:36.632769 14715 tablet_replica.cc:330] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: stopping tablet replica
I20260806 06:46:36.632858 14715 raft_consensus.cc:2229] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:36.632953 14715 raft_consensus.cc:2258] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.633103 14713 tablet_replica.cc:330] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: stopping tablet replica
I20260806 06:46:36.633186 14713 raft_consensus.cc:2229] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:36.633275 14713 raft_consensus.cc:2258] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.633726 14713 ts_tablet_manager.cc:1905] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.633746 14304 tablet_service.cc:1467] Processing CreateTablet for tablet bc3194b6f19843339e384d922a01f2ee (DEFAULT_TABLE table=test-table [id=983d81bf64d54bffb2c1ec0124bd3507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.633864 14304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3194b6f19843339e384d922a01f2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.633970 14715 ts_tablet_manager.cc:1905] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.635298 14449 tablet_service.cc:1467] Processing CreateTablet for tablet bc3194b6f19843339e384d922a01f2ee (DEFAULT_TABLE table=test-table [id=983d81bf64d54bffb2c1ec0124bd3507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.635425 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3194b6f19843339e384d922a01f2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.635545 14713 ts_tablet_manager.cc:1918] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:36.635620 14713 log.cc:1196] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/54997d9a2f184b2bb9175ffade9a08d1
I20260806 06:46:36.635910 14713 ts_tablet_manager.cc:1939] T 54997d9a2f184b2bb9175ffade9a08d1 P e791a9d28aa24be4afc48c95d82e19e3: Deleting consensus metadata
I20260806 06:46:36.636272 14715 ts_tablet_manager.cc:1918] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:36.636372 14715 log.cc:1196] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/54997d9a2f184b2bb9175ffade9a08d1
I20260806 06:46:36.636601 14715 ts_tablet_manager.cc:1939] T 54997d9a2f184b2bb9175ffade9a08d1 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting consensus metadata
I20260806 06:46:36.636739 14593 tablet_service.cc:1467] Processing CreateTablet for tablet bc3194b6f19843339e384d922a01f2ee (DEFAULT_TABLE table=test-table [id=983d81bf64d54bffb2c1ec0124bd3507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.636868 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc3194b6f19843339e384d922a01f2ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.638059 14690 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Bootstrap starting.
I20260806 06:46:36.638067 14691 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Bootstrap starting.
I20260806 06:46:36.638448 14690 tablet_bootstrap.cc:654] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.638499 14691 tablet_bootstrap.cc:654] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.638814 14694 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Bootstrap starting.
I20260806 06:46:36.639067 14690 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: No bootstrap required, opened a new log
I20260806 06:46:36.639143 14690 ts_tablet_manager.cc:1397] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:36.639236 14694 tablet_bootstrap.cc:654] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.639302 14690 raft_consensus.cc:348] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.639372 14690 raft_consensus.cc:374] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.639408 14690 raft_consensus.cc:729] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.639475 14690 consensus_queue.cc:260] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.639577 14690 ts_tablet_manager.cc:1428] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.639657 14144 catalog_manager.cc:4841] TS e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911): tablet 54997d9a2f184b2bb9175ffade9a08d1 (table test-table [id=b33426ceb91340e49d68a515e441d3dc]) successfully deleted
I20260806 06:46:36.639954 14146 catalog_manager.cc:4841] TS 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889): tablet 54997d9a2f184b2bb9175ffade9a08d1 (table test-table [id=b33426ceb91340e49d68a515e441d3dc]) successfully deleted
I20260806 06:46:36.640295 14694 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: No bootstrap required, opened a new log
I20260806 06:46:36.640339 14694 ts_tablet_manager.cc:1397] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:36.640487 14694 raft_consensus.cc:348] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.640566 14694 raft_consensus.cc:374] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.640589 14694 raft_consensus.cc:729] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55a1b06905443e2910f7494d17c90aa, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.640632 14694 consensus_queue.cc:260] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.640724 14694 ts_tablet_manager.cc:1428] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:36.640925 14714 tablet_replica.cc:330] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: stopping tablet replica
I20260806 06:46:36.641016 14714 raft_consensus.cc:2229] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:36.641140 14714 raft_consensus.cc:2258] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.641520 14714 ts_tablet_manager.cc:1905] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.641746 14691 tablet_bootstrap.cc:492] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: No bootstrap required, opened a new log
I20260806 06:46:36.641809 14691 ts_tablet_manager.cc:1397] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 06:46:36.642103 14691 raft_consensus.cc:348] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.642181 14691 raft_consensus.cc:374] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.642205 14691 raft_consensus.cc:729] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.642257 14691 consensus_queue.cc:260] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.642354 14691 ts_tablet_manager.cc:1428] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:36.643167 14714 ts_tablet_manager.cc:1918] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:36.643241 14714 log.cc:1196] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/54997d9a2f184b2bb9175ffade9a08d1
I20260806 06:46:36.643435 14696 raft_consensus.cc:482] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:36.643514 14714 ts_tablet_manager.cc:1939] T 54997d9a2f184b2bb9175ffade9a08d1 P a55a1b06905443e2910f7494d17c90aa: Deleting consensus metadata
I20260806 06:46:36.644172 14696 raft_consensus.cc:504] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.644444 14696 leader_election.cc:288] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911)
I20260806 06:46:36.650207 14147 catalog_manager.cc:4841] TS a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429): tablet 54997d9a2f184b2bb9175ffade9a08d1 (table test-table [id=b33426ceb91340e49d68a515e441d3dc]) successfully deleted
I20260806 06:46:36.653632 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.650403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:34911 (local address 127.6.57.65:43955)
0806 06:46:36.650577 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.650586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.650661 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.650955 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.650956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.650958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.651019 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.651022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.651727 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.651728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.652572 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.652575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.653356 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.653357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.653427 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.653431 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.653507 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.653547 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260806 06:46:36.653646 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.650150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:36145 (local address 127.6.57.66:46429)
0806 06:46:36.650198 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.650200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.650240 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:46:36.650305 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.650306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.650315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.650365 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.650648 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.651298 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.652769 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.653293 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.653357 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.653366 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.653542 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.653586 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:36.653609 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.650498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:43955 (local address 127.6.57.67:34911)
0806 06:46:36.650561 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.650563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.650635 (+ 72us) server_negotiation.cc:407] Connection header received
0806 06:46:36.650856 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.650857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.650866 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.650896 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:36.651095 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.651644 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.652663 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.653224 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.653292 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.653303 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.653518 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.653551 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:46:36.653906 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.650051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46429 (local address 127.6.57.65:36145)
0806 06:46:36.650187 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.650198 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.650255 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.650410 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.650411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.650413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:36.650536 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.650539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.651382 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.651383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.652683 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.652686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.653372 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.653373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.653447 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.653452 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.653789 (+ 337us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.653830 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":66,"threads_started":1}
I20260806 06:46:36.654044 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3194b6f19843339e384d922a01f2ee" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3" is_pre_election: true
I20260806 06:46:36.654121 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3194b6f19843339e384d922a01f2ee" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa" is_pre_election: true
I20260806 06:46:36.654175 14613 raft_consensus.cc:2454] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:36.654213 14469 raft_consensus.cc:2454] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:36.654403 14256 leader_election.cc:302] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, a55a1b06905443e2910f7494d17c90aa; no voters:
I20260806 06:46:36.654583 14696 raft_consensus.cc:2781] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:36.654626 14696 raft_consensus.cc:482] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:36.654647 14696 raft_consensus.cc:3037] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.655288 14696 raft_consensus.cc:504] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.655438 14696 leader_election.cc:288] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Requested vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911)
I20260806 06:46:36.655560 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3194b6f19843339e384d922a01f2ee" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa"
I20260806 06:46:36.655629 14469 raft_consensus.cc:3037] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.655625 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc3194b6f19843339e384d922a01f2ee" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3"
I20260806 06:46:36.655691 14613 raft_consensus.cc:3037] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.656392 14469 raft_consensus.cc:2454] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:36.656528 14613 raft_consensus.cc:2454] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:36.656594 14256 leader_election.cc:302] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, a55a1b06905443e2910f7494d17c90aa; no voters:
I20260806 06:46:36.656672 14696 raft_consensus.cc:2781] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:36.656823 14696 raft_consensus.cc:686] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 LEADER]: Becoming Leader. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Running, Role: LEADER
I20260806 06:46:36.656895 14696 consensus_queue.cc:237] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.657555 14160 catalog_manager.cc:5495] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65). New cstate: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } health_report { overall_health: UNKNOWN } } }
W20260806 06:46:36.673382 14663 tablet.cc:2369] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998795, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.126@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.686918 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.679157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42585 (local address 127.0.0.1:34816)
0806 06:46:36.679325 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.679336 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.679399 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.679536 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.679537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.679819 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:36.679866 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.679867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.680802 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.680803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.681460 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.681463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.681941 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.681943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.681991 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.682698 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:36.682703 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:36.684166 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:36.685860 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.685867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.685869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.686095 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.686297 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:36.686297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:36.686297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:36.686354 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:36.686659 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:36.686666 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:36.686708 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.686760 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.686848 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260806 06:46:36.687026 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.679238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.6.57.126:42585)
0806 06:46:36.679306 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.679309 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.679401 (+ 92us) server_negotiation.cc:407] Connection header received
0806 06:46:36.679444 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.679445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.679456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.679523 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.679939 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.680714 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.681537 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.681841 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.681917 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.684252 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.684253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.684254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.684261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.685769 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.686166 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.686168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.686168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.686230 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.686415 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.686416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.686416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.686587 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.686652 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.686807 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.686912 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":41}
I20260806 06:46:36.689038 14160 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:34816:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 06:46:36.689270 14160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:36.690754 14593 tablet_service.cc:1467] Processing CreateTablet for tablet 9d6e967b919d4d2b8d36368bd1a33589 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d11fd9db7c74181b82de9d771c6dbc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:36.690845 14304 tablet_service.cc:1467] Processing CreateTablet for tablet 9d6e967b919d4d2b8d36368bd1a33589 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d11fd9db7c74181b82de9d771c6dbc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:36.690833 14449 tablet_service.cc:1467] Processing CreateTablet for tablet 9d6e967b919d4d2b8d36368bd1a33589 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9d11fd9db7c74181b82de9d771c6dbc1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 06:46:36.690913 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e967b919d4d2b8d36368bd1a33589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.690984 14304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e967b919d4d2b8d36368bd1a33589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.690977 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d6e967b919d4d2b8d36368bd1a33589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.692480 14694 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: Bootstrap starting.
I20260806 06:46:36.692523 14690 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: Bootstrap starting.
I20260806 06:46:36.692806 14690 tablet_bootstrap.cc:654] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.692891 14694 tablet_bootstrap.cc:654] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.693099 14691 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Bootstrap starting.
I20260806 06:46:36.693450 14691 tablet_bootstrap.cc:654] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.694036 14691 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: No bootstrap required, opened a new log
I20260806 06:46:36.694085 14691 ts_tablet_manager.cc:1397] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:36.694128 14690 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: No bootstrap required, opened a new log
I20260806 06:46:36.694172 14690 ts_tablet_manager.cc:1397] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:46:36.694231 14691 raft_consensus.cc:348] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.694298 14691 raft_consensus.cc:374] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.694319 14691 raft_consensus.cc:729] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.694296 14690 raft_consensus.cc:348] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.694356 14690 raft_consensus.cc:374] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.694380 14690 raft_consensus.cc:729] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.694363 14691 consensus_queue.cc:260] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.694419 14690 consensus_queue.cc:260] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.694450 14691 ts_tablet_manager.cc:1428] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:36.694563 14708 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.694612 14696 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.694644 14708 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.694674 14696 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.694993 14690 ts_tablet_manager.cc:1428] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:36.695485 14694 tablet_bootstrap.cc:492] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: No bootstrap required, opened a new log
I20260806 06:46:36.695535 14694 ts_tablet_manager.cc:1397] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:46:36.695677 14694 raft_consensus.cc:348] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.695745 14694 raft_consensus.cc:374] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.695765 14694 raft_consensus.cc:729] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55a1b06905443e2910f7494d17c90aa, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.695803 14694 consensus_queue.cc:260] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.695885 14694 ts_tablet_manager.cc:1428] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.695969 14698 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.696149 14698 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.872417 14696 raft_consensus.cc:482] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:36.872530 14696 raft_consensus.cc:504] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.872745 14696 leader_election.cc:288] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911), a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429)
I20260806 06:46:36.872946 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d6e967b919d4d2b8d36368bd1a33589" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3" is_pre_election: true
I20260806 06:46:36.873070 14613 raft_consensus.cc:2454] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:36.873055 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d6e967b919d4d2b8d36368bd1a33589" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa" is_pre_election: true
I20260806 06:46:36.873164 14469 raft_consensus.cc:2454] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:36.873241 14253 leader_election.cc:302] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:36.873332 14696 raft_consensus.cc:2781] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:36.873368 14696 raft_consensus.cc:482] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:36.873382 14696 raft_consensus.cc:3037] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.874119 14696 raft_consensus.cc:504] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.874246 14696 leader_election.cc:288] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Requested vote from peers e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911), a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429)
I20260806 06:46:36.874388 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d6e967b919d4d2b8d36368bd1a33589" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3"
I20260806 06:46:36.874460 14613 raft_consensus.cc:3037] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.874512 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d6e967b919d4d2b8d36368bd1a33589" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa"
I20260806 06:46:36.874624 14469 raft_consensus.cc:3037] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:36.875211 14613 raft_consensus.cc:2454] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:36.875386 14253 leader_election.cc:302] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:36.875504 14696 raft_consensus.cc:2781] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:36.875548 14696 raft_consensus.cc:686] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 LEADER]: Becoming Leader. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Running, Role: LEADER
I20260806 06:46:36.875532 14469 raft_consensus.cc:2454] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:36.875595 14696 consensus_queue.cc:237] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } }
I20260806 06:46:36.875945 14721 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: TxnStatusTablet state changed. Reason: New leader 0c94f7604b0940a7a1c7b85ec80b9fdf. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.876070 14721 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:36.876237 14161 catalog_manager.cc:5495] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65). New cstate: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:36.876284 14735 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 06:46:36.876411 14735 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 06:46:36.925446 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.922545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44889 (local address 127.0.0.1:39408)
0806 06:46:36.922610 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.922622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.922681 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.922859 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.922860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.922861 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:36.922908 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.922909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.923793 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.923794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.924370 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.924373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.924885 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.924886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.924946 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.924961 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.925277 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.925285 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.925342 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.925386 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:46:36.925448 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.922571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.6.57.65:44889)
0806 06:46:36.922621 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.922623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.922667 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:46:36.922745 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.922746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.922769 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.922848 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:36.922952 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.923701 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.924473 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.924773 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.924844 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.925066 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.925207 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.925281 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.925348 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.925372 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:46:36.926687 14469 raft_consensus.cc:1261] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.926765 14613 raft_consensus.cc:1261] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.926975 14721 consensus_queue.cc:1035] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.927142 14696 consensus_queue.cc:1035] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.927724 14698 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: TxnStatusTablet state changed. Reason: New leader 0c94f7604b0940a7a1c7b85ec80b9fdf. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.927827 14698 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.928311 14708 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: TxnStatusTablet state changed. Reason: New leader 0c94f7604b0940a7a1c7b85ec80b9fdf. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.928418 14708 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.928788 14721 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.928881 14721 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:36.928943 14721 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.929000 14721 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P 0c94f7604b0940a7a1c7b85ec80b9fdf: This TxnStatusTablet replica's current role is: LEADER
I20260806 06:46:36.929133 14703 mvcc.cc:205] Tried to move back new op lower bound from 7315451072209543168 to 7315451072002748416. Current Snapshot: MvccSnapshot[applied={T|T < 7315451072209543168}]
I20260806 06:46:36.929194 14708 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.929262 14708 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P e791a9d28aa24be4afc48c95d82e19e3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.929698 14698 tablet_replica.cc:439] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } }
I20260806 06:46:36.929782 14698 tablet_replica.cc:442] T 9d6e967b919d4d2b8d36368bd1a33589 P a55a1b06905443e2910f7494d17c90aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 06:46:36.933143 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.930750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.6.57.65:44889)
0806 06:46:36.930919 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.930921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.930929 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:36.930948 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.930949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.930958 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.931009 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:36.931182 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.931601 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.932600 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.932766 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.932813 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.932976 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.932997 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.933031 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.933085 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.933099 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260806 06:46:36.938048 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.935660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:46273 (local address 127.6.57.65:44889)
0806 06:46:36.935699 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.935701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.935789 (+ 88us) server_negotiation.cc:407] Connection header received
0806 06:46:36.935861 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.935861 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.935868 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.935910 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:36.936016 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.936415 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.937410 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.937576 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.937619 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.937840 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.937862 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.937899 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.937990 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.938002 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:46:36.938237 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.935565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44889 (local address 127.6.57.65:46273)
0806 06:46:36.935729 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:36.935740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:36.935804 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:36.935917 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:36.935917 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:36.935919 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 06:46:36.935958 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.935959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.936523 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.936524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.937331 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:36.937334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.937651 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.937652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:36.937709 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.937724 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 06:46:36.937928 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 06:46:36.937931 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:36.937989 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:46:36.938034 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260806 06:46:36.940181 14469 raft_consensus.cc:1261] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.940186 14613 raft_consensus.cc:1261] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:36.940377 14696 consensus_queue.cc:1035] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.940469 14721 consensus_queue.cc:1035] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:36.952147 14160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:36.952247 14160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:36.952903 14160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Sending DeleteTablet for 3 replicas of tablet bc3194b6f19843339e384d922a01f2ee
I20260806 06:46:36.953182 14303 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3194b6f19843339e384d922a01f2ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:39406
I20260806 06:46:36.953186 14593 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3194b6f19843339e384d922a01f2ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43978
I20260806 06:46:36.953276 14715 tablet_replica.cc:330] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: stopping tablet replica
I20260806 06:46:36.953325 14715 raft_consensus.cc:2229] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:36.953408 14715 raft_consensus.cc:2258] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.953497 14713 tablet_replica.cc:330] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: stopping tablet replica
I20260806 06:46:36.953557 14713 raft_consensus.cc:2229] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:36.953595 14713 raft_consensus.cc:2258] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.955667 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet bc3194b6f19843339e384d922a01f2ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:36 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 06:46:36.955758 14714 tablet_replica.cc:330] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: stopping tablet replica
I20260806 06:46:36.955801 14714 raft_consensus.cc:2229] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:36.955793 14715 ts_tablet_manager.cc:1905] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.955833 14714 raft_consensus.cc:2258] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:36.956651 14713 ts_tablet_manager.cc:1905] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.957717 14714 ts_tablet_manager.cc:1905] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:36.958354 14713 ts_tablet_manager.cc:1918] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:36.958420 14713 log.cc:1196] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/bc3194b6f19843339e384d922a01f2ee
I20260806 06:46:36.958802 14713 ts_tablet_manager.cc:1939] T bc3194b6f19843339e384d922a01f2ee P e791a9d28aa24be4afc48c95d82e19e3: Deleting consensus metadata
I20260806 06:46:36.959255 14144 catalog_manager.cc:4841] TS e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911): tablet bc3194b6f19843339e384d922a01f2ee (table test-table [id=983d81bf64d54bffb2c1ec0124bd3507]) successfully deleted
I20260806 06:46:36.959440 14715 ts_tablet_manager.cc:1918] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:36.959509 14715 log.cc:1196] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/bc3194b6f19843339e384d922a01f2ee
I20260806 06:46:36.959736 14715 ts_tablet_manager.cc:1939] T bc3194b6f19843339e384d922a01f2ee P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting consensus metadata
I20260806 06:46:36.959731 14714 ts_tablet_manager.cc:1918] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:36.959789 14714 log.cc:1196] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/bc3194b6f19843339e384d922a01f2ee
I20260806 06:46:36.960006 14714 ts_tablet_manager.cc:1939] T bc3194b6f19843339e384d922a01f2ee P a55a1b06905443e2910f7494d17c90aa: Deleting consensus metadata
I20260806 06:46:36.960067 14146 catalog_manager.cc:4841] TS 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889): tablet bc3194b6f19843339e384d922a01f2ee (table test-table [id=983d81bf64d54bffb2c1ec0124bd3507]) successfully deleted
I20260806 06:46:36.960533 14147 catalog_manager.cc:4841] TS a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429): tablet bc3194b6f19843339e384d922a01f2ee (table test-table [id=983d81bf64d54bffb2c1ec0124bd3507]) successfully deleted
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:46:36.980566 14214 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:36.974868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.6.57.126:42585)
0806 06:46:36.974929 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.974933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.975085 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:46:36.975168 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.975169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.975181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.975255 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.975850 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.976347 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.977227 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.977425 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.977500 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.980502 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34818: BlockingRecv error: recv got EOF from 127.0.0.1:34818 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 06:46:36 dist-test-slave-5l3k krb5kdc[14114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM
I20260806 06:46:36.988948 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:36.981550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.6.57.126:42585)
0806 06:46:36.981584 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:36.981586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:36.981595 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:36.981645 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:36.981646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:36.981658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:36.981703 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:36.982334 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.982876 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.983851 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:36.984074 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:36.984126 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:36.986471 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:36.986472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:36.986473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:36.986512 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:36.987867 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.988209 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.988210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.988211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.988268 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:36.988470 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:36.988470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:36.988471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:36.988576 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:36.988672 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:36.988783 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:46:36.988853 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:46:36.990510 14160 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:36.990686 14160 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:36.992187 14303 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 (DEFAULT_TABLE table=test-table [id=26827fd830a4457e8b770cd1cf47b5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.992233 14449 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 (DEFAULT_TABLE table=test-table [id=26827fd830a4457e8b770cd1cf47b5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.992254 14593 tablet_service.cc:1467] Processing CreateTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 (DEFAULT_TABLE table=test-table [id=26827fd830a4457e8b770cd1cf47b5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:36.992331 14303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9f698f77da4a23a69c91b10035a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.992358 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9f698f77da4a23a69c91b10035a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.992358 14593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f9f698f77da4a23a69c91b10035a5c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:36.993703 14690 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Bootstrap starting.
I20260806 06:46:36.993955 14694 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Bootstrap starting.
I20260806 06:46:36.994071 14690 tablet_bootstrap.cc:654] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.994359 14694 tablet_bootstrap.cc:654] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.994704 14690 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: No bootstrap required, opened a new log
I20260806 06:46:36.994771 14691 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Bootstrap starting.
I20260806 06:46:36.994779 14690 ts_tablet_manager.cc:1397] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:36.994908 14690 raft_consensus.cc:348] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.994987 14690 raft_consensus.cc:374] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.995023 14690 raft_consensus.cc:729] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.995061 14690 consensus_queue.cc:260] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.995177 14690 ts_tablet_manager.cc:1428] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:36.995174 14691 tablet_bootstrap.cc:654] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:36.996146 14694 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: No bootstrap required, opened a new log
I20260806 06:46:36.996201 14694 ts_tablet_manager.cc:1397] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:46:36.996342 14694 raft_consensus.cc:348] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.996416 14694 raft_consensus.cc:374] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.996438 14694 raft_consensus.cc:729] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55a1b06905443e2910f7494d17c90aa, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.996510 14694 consensus_queue.cc:260] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.996590 14694 ts_tablet_manager.cc:1428] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:36.996928 14691 tablet_bootstrap.cc:492] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: No bootstrap required, opened a new log
I20260806 06:46:36.996979 14691 ts_tablet_manager.cc:1397] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:36.997125 14691 raft_consensus.cc:348] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.997200 14691 raft_consensus.cc:374] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:36.997221 14691 raft_consensus.cc:729] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Initialized, Role: FOLLOWER
I20260806 06:46:36.997263 14691 consensus_queue.cc:260] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:36.997357 14691 ts_tablet_manager.cc:1428] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:37.082149 14708 raft_consensus.cc:482] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:37.082306 14708 raft_consensus.cc:504] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.082537 14708 leader_election.cc:288] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889), a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429)
I20260806 06:46:37.082840 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9f698f77da4a23a69c91b10035a5c8" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" is_pre_election: true
I20260806 06:46:37.082854 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9f698f77da4a23a69c91b10035a5c8" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa" is_pre_election: true
I20260806 06:46:37.082940 14324 raft_consensus.cc:2454] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 0.
I20260806 06:46:37.082940 14469 raft_consensus.cc:2454] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 0.
I20260806 06:46:37.083097 14545 leader_election.cc:302] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55a1b06905443e2910f7494d17c90aa, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:37.083204 14708 raft_consensus.cc:2781] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:37.083236 14708 raft_consensus.cc:482] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:37.083280 14708 raft_consensus.cc:3037] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.084003 14708 raft_consensus.cc:504] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.084141 14708 leader_election.cc:288] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 election: Requested vote from peers 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889), a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429)
I20260806 06:46:37.084280 14324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9f698f77da4a23a69c91b10035a5c8" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf"
I20260806 06:46:37.084363 14324 raft_consensus.cc:3037] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.084335 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f9f698f77da4a23a69c91b10035a5c8" candidate_uuid: "e791a9d28aa24be4afc48c95d82e19e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa"
I20260806 06:46:37.084401 14469 raft_consensus.cc:3037] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.084956 14469 raft_consensus.cc:2454] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 1.
I20260806 06:46:37.085114 14545 leader_election.cc:302] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55a1b06905443e2910f7494d17c90aa, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:37.085188 14708 raft_consensus.cc:2781] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:37.085194 14324 raft_consensus.cc:2454] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e791a9d28aa24be4afc48c95d82e19e3 in term 1.
I20260806 06:46:37.085227 14708 raft_consensus.cc:686] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 LEADER]: Becoming Leader. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Running, Role: LEADER
I20260806 06:46:37.085286 14708 consensus_queue.cc:237] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.086062 14161 catalog_manager.cc:5495] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "e791a9d28aa24be4afc48c95d82e19e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } health_report { overall_health: HEALTHY } } }
I20260806 06:46:37.092844 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:37.090283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.6.57.67:34911)
0806 06:46:37.090327 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:37.090329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:37.090376 (+ 47us) server_negotiation.cc:407] Connection header received
0806 06:46:37.090461 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:37.090462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:37.090506 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:37.090569 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:37.090721 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:37.091205 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:37.092068 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:37.092263 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:37.092380 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:37.092547 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:37.092607 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:37.092673 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:37.092738 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:46:37.092761 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:46:37.093593 14469 raft_consensus.cc:1261] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Refusing update from remote peer e791a9d28aa24be4afc48c95d82e19e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:37.093648 14324 raft_consensus.cc:1261] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Refusing update from remote peer e791a9d28aa24be4afc48c95d82e19e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:37.093849 14708 consensus_queue.cc:1035] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:37.093961 14700 consensus_queue.cc:1035] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:37.096909 14161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:37.097014 14161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:37.097829 14161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Sending DeleteTablet for 3 replicas of tablet 3f9f698f77da4a23a69c91b10035a5c8
I20260806 06:46:37.098109 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 06:46:37.098132 14303 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:39406
I20260806 06:46:37.098217 14714 tablet_replica.cc:330] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: stopping tablet replica
I20260806 06:46:37.098223 14593 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43978
I20260806 06:46:37.098278 14714 raft_consensus.cc:2229] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:37.098326 14714 raft_consensus.cc:2258] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.098310 14592 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f9f698f77da4a23a69c91b10035a5c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43978
I20260806 06:46:37.098382 14713 tablet_replica.cc:330] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: stopping tablet replica
I20260806 06:46:37.098407 14715 tablet_replica.cc:330] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: stopping tablet replica
I20260806 06:46:37.098428 14713 raft_consensus.cc:2229] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:37.098531 14713 raft_consensus.cc:2258] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.098472 14715 raft_consensus.cc:2229] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:37.098587 14715 raft_consensus.cc:2258] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.098937 14714 ts_tablet_manager.cc:1905] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:37.098958 14715 ts_tablet_manager.cc:1905] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:37.099054 14161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:37.099205 14161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:37.099540 14713 ts_tablet_manager.cc:1905] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:37.100812 14714 ts_tablet_manager.cc:1918] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:37.100888 14714 log.cc:1196] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/3f9f698f77da4a23a69c91b10035a5c8
I20260806 06:46:37.101200 14714 ts_tablet_manager.cc:1939] T 3f9f698f77da4a23a69c91b10035a5c8 P a55a1b06905443e2910f7494d17c90aa: Deleting consensus metadata
I20260806 06:46:37.101227 14713 ts_tablet_manager.cc:1918] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:37.101284 14713 log.cc:1196] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/3f9f698f77da4a23a69c91b10035a5c8
I20260806 06:46:37.101522 14713 ts_tablet_manager.cc:1939] T 3f9f698f77da4a23a69c91b10035a5c8 P e791a9d28aa24be4afc48c95d82e19e3: Deleting consensus metadata
I20260806 06:46:37.101537 14715 ts_tablet_manager.cc:1918] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:37.101588 14715 log.cc:1196] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/3f9f698f77da4a23a69c91b10035a5c8
I20260806 06:46:37.101673 14147 catalog_manager.cc:4841] TS a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429): tablet 3f9f698f77da4a23a69c91b10035a5c8 (table test-table [id=26827fd830a4457e8b770cd1cf47b5c3]) successfully deleted
I20260806 06:46:37.101827 14715 ts_tablet_manager.cc:1939] T 3f9f698f77da4a23a69c91b10035a5c8 P 0c94f7604b0940a7a1c7b85ec80b9fdf: Deleting consensus metadata
I20260806 06:46:37.102029 14144 catalog_manager.cc:4841] TS e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911): tablet 3f9f698f77da4a23a69c91b10035a5c8 (table test-table [id=26827fd830a4457e8b770cd1cf47b5c3]) successfully deleted
I20260806 06:46:37.102150 14146 catalog_manager.cc:4841] TS 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889): tablet 3f9f698f77da4a23a69c91b10035a5c8 (table test-table [id=26827fd830a4457e8b770cd1cf47b5c3]) successfully deleted
W20260806 06:46:37.102175 14144 catalog_manager.cc:4808] TS e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911): delete failed for tablet 3f9f698f77da4a23a69c91b10035a5c8 because the tablet was not found. No further retry: Not found: Tablet not found: 3f9f698f77da4a23a69c91b10035a5c8
I20260806 06:46:37.102458 14303 tablet_service.cc:1467] Processing CreateTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a (DEFAULT_TABLE table=test-table [id=640a2a30e7714f74906dfce904261349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:37.102460 14449 tablet_service.cc:1467] Processing CreateTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a (DEFAULT_TABLE table=test-table [id=640a2a30e7714f74906dfce904261349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:37.102599 14303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cdd11eb59844c096f9d13d99a1bc3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:37.102622 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cdd11eb59844c096f9d13d99a1bc3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:37.103454 14694 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Bootstrap starting.
I20260806 06:46:37.103780 14691 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: Bootstrap starting.
I20260806 06:46:37.104123 14691 tablet_bootstrap.cc:654] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:37.104199 14694 tablet_bootstrap.cc:654] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:37.104209 14592 tablet_service.cc:1467] Processing CreateTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a (DEFAULT_TABLE table=test-table [id=640a2a30e7714f74906dfce904261349]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:37.104326 14592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cdd11eb59844c096f9d13d99a1bc3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:37.104883 14694 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: No bootstrap required, opened a new log
I20260806 06:46:37.104920 14694 ts_tablet_manager.cc:1397] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:37.105055 14694 raft_consensus.cc:348] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.105103 14694 raft_consensus.cc:374] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:37.105119 14694 raft_consensus.cc:729] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55a1b06905443e2910f7494d17c90aa, State: Initialized, Role: FOLLOWER
I20260806 06:46:37.105181 14694 consensus_queue.cc:260] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.105280 14694 ts_tablet_manager.cc:1428] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:37.105593 14690 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Bootstrap starting.
I20260806 06:46:37.105929 14690 tablet_bootstrap.cc:654] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:37.106303 14691 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: No bootstrap required, opened a new log
I20260806 06:46:37.106349 14691 ts_tablet_manager.cc:1397] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:46:37.106465 14691 raft_consensus.cc:348] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.106552 14691 raft_consensus.cc:374] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:37.106580 14691 raft_consensus.cc:729] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Initialized, Role: FOLLOWER
I20260806 06:46:37.106622 14691 consensus_queue.cc:260] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.106700 14691 ts_tablet_manager.cc:1428] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:37.106958 14690 tablet_bootstrap.cc:492] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: No bootstrap required, opened a new log
I20260806 06:46:37.107030 14690 ts_tablet_manager.cc:1397] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:46:37.107172 14690 raft_consensus.cc:348] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.107245 14690 raft_consensus.cc:374] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:37.107273 14690 raft_consensus.cc:729] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e791a9d28aa24be4afc48c95d82e19e3, State: Initialized, Role: FOLLOWER
I20260806 06:46:37.107326 14690 consensus_queue.cc:260] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.107419 14690 ts_tablet_manager.cc:1428] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:37.143864 14745 raft_consensus.cc:482] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:37.143968 14745 raft_consensus.cc:504] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.144141 14745 leader_election.cc:288] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911)
I20260806 06:46:37.144330 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cdd11eb59844c096f9d13d99a1bc3a" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa" is_pre_election: true
I20260806 06:46:37.144371 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cdd11eb59844c096f9d13d99a1bc3a" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3" is_pre_election: true
I20260806 06:46:37.144443 14469 raft_consensus.cc:2454] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:37.144487 14613 raft_consensus.cc:2454] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 0.
I20260806 06:46:37.144666 14253 leader_election.cc:302] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, e791a9d28aa24be4afc48c95d82e19e3; no voters:
I20260806 06:46:37.144819 14745 raft_consensus.cc:2781] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:37.144870 14745 raft_consensus.cc:482] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:37.144886 14745 raft_consensus.cc:3037] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.145587 14745 raft_consensus.cc:504] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.145733 14745 leader_election.cc:288] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Requested vote from peers a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429), e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911)
I20260806 06:46:37.145908 14469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cdd11eb59844c096f9d13d99a1bc3a" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55a1b06905443e2910f7494d17c90aa"
I20260806 06:46:37.145908 14613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cdd11eb59844c096f9d13d99a1bc3a" candidate_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e791a9d28aa24be4afc48c95d82e19e3"
I20260806 06:46:37.145988 14469 raft_consensus.cc:3037] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.145988 14613 raft_consensus.cc:3037] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:37.146790 14469 raft_consensus.cc:2454] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:37.146790 14613 raft_consensus.cc:2454] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c94f7604b0940a7a1c7b85ec80b9fdf in term 1.
I20260806 06:46:37.146991 14256 leader_election.cc:302] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c94f7604b0940a7a1c7b85ec80b9fdf, a55a1b06905443e2910f7494d17c90aa; no voters:
I20260806 06:46:37.147145 14745 raft_consensus.cc:2781] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:37.147213 14745 raft_consensus.cc:686] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 LEADER]: Becoming Leader. State: Replica: 0c94f7604b0940a7a1c7b85ec80b9fdf, State: Running, Role: LEADER
I20260806 06:46:37.147269 14745 consensus_queue.cc:237] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } }
I20260806 06:46:37.147881 14161 catalog_manager.cc:5495] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65). New cstate: current_term: 1 leader_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c94f7604b0940a7a1c7b85ec80b9fdf" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:37.157907 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:37.155871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.6.57.65:44889)
0806 06:46:37.155914 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:37.155916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:37.155976 (+ 60us) server_negotiation.cc:407] Connection header received
0806 06:46:37.156029 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:37.156030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:37.156044 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:37.156089 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:37.156216 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:37.156670 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:37.157360 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:37.157540 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:37.157595 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:37.157703 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:37.157739 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:37.157794 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:37.157841 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 06:46:37.157857 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:37.160005 14613 raft_consensus.cc:1261] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:37.160027 14469 raft_consensus.cc:1261] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Refusing update from remote peer 0c94f7604b0940a7a1c7b85ec80b9fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:37.160212 14696 consensus_queue.cc:1035] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e791a9d28aa24be4afc48c95d82e19e3" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:37.160337 14721 consensus_queue.cc:1035] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55a1b06905443e2910f7494d17c90aa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:37.173765 14161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:37.173906 14161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:37.174681 14161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f642285ac1234dfb8da611b54cd7cdf2: Sending DeleteTablet for 3 replicas of tablet 78cdd11eb59844c096f9d13d99a1bc3a
I20260806 06:46:37.174981 14592 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:43978
I20260806 06:46:37.175091 14303 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:39406
I20260806 06:46:37.175140 14713 tablet_replica.cc:330] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: stopping tablet replica
I20260806 06:46:37.175215 14713 raft_consensus.cc:2229] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:37.175263 14713 raft_consensus.cc:2258] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.175352 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cdd11eb59844c096f9d13d99a1bc3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:37 UTC) from {username='oryx', principal='oryx/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 06:46:37.175631 14715 tablet_replica.cc:330] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf: stopping tablet replica
I20260806 06:46:37.175702 14715 raft_consensus.cc:2229] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:37.175773 14715 raft_consensus.cc:2258] T 78cdd11eb59844c096f9d13d99a1bc3a P 0c94f7604b0940a7a1c7b85ec80b9fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.175850 14713 ts_tablet_manager.cc:1905] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:37.175860 14714 tablet_replica.cc:330] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: stopping tablet replica
I20260806 06:46:37.175935 14714 raft_consensus.cc:2229] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:37.175984 14714 raft_consensus.cc:2258] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:37.176268 14714 ts_tablet_manager.cc:1905] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:37.176326 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14236
I20260806 06:46:37.177651 14713 ts_tablet_manager.cc:1918] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:37.177738 14713 log.cc:1196] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/78cdd11eb59844c096f9d13d99a1bc3a
I20260806 06:46:37.178169 14714 ts_tablet_manager.cc:1918] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 06:46:37.178234 14714 log.cc:1196] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/78cdd11eb59844c096f9d13d99a1bc3a
I20260806 06:46:37.178467 14714 ts_tablet_manager.cc:1939] T 78cdd11eb59844c096f9d13d99a1bc3a P a55a1b06905443e2910f7494d17c90aa: Deleting consensus metadata
I20260806 06:46:37.179693 14713 ts_tablet_manager.cc:1939] T 78cdd11eb59844c096f9d13d99a1bc3a P e791a9d28aa24be4afc48c95d82e19e3: Deleting consensus metadata
I20260806 06:46:37.180032 14144 catalog_manager.cc:4841] TS e791a9d28aa24be4afc48c95d82e19e3 (127.6.57.67:34911): tablet 78cdd11eb59844c096f9d13d99a1bc3a (table test-table [id=640a2a30e7714f74906dfce904261349]) successfully deleted
I20260806 06:46:37.180032 14147 catalog_manager.cc:4841] TS a55a1b06905443e2910f7494d17c90aa (127.6.57.66:46429): tablet 78cdd11eb59844c096f9d13d99a1bc3a (table test-table [id=640a2a30e7714f74906dfce904261349]) successfully deleted
W20260806 06:46:37.185588 14146 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:44889 (error 108)
I20260806 06:46:37.185891 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14380
I20260806 06:46:37.186085 14775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:37.185904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44889 (local address 127.0.0.1:39432)
0806 06:46:37.185998 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:37.186039 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:44889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":51,"threads_started":1}
W20260806 06:46:37.187805 14146 catalog_manager.cc:4584] TS 0c94f7604b0940a7a1c7b85ec80b9fdf (127.6.57.65:44889): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78cdd11eb59844c096f9d13d99a1bc3a: Network error: Client connection negotiation failed: client connection to 127.6.57.65:44889: connect: Connection refused (error 111)
I20260806 06:46:37.191710 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14526
I20260806 06:46:37.198757 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14128
2026-08-06T06:46:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2670 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:37 dist-test-slave-5l3k krb5kdc[14778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:37 dist-test-slave-5l3k krb5kdc[14778](info): set up 2 sockets
Aug 06 06:46:37 dist-test-slave-5l3k krb5kdc[14778](info): commencing operation
krb5kdc: starting...
W20260806 06:46:38.293411 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:38 dist-test-slave-5l3k krb5kdc[14778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:38.412698 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37135
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:38.493834 14791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:38.494063 14791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:38.494095 14791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:38.495276 14791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:38.495343 14791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:38.495368 14791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:38.495388 14791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:38.495424 14791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:38.496914 14791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:37135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:38.497192 14791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:38.497407 14791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:38.502163 14799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:38.502164 14797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:38.502285 14791 server_base.cc:1034] running on GCE node
W20260806 06:46:38.502164 14796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:38.502686 14791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:38.503001 14791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:38.504144 14791 hybrid_clock.cc:648] HybridClock initialized: now 1785998798504134 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T06:46:38Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1308 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:38 dist-test-slave-5l3k krb5kdc[14804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:38 dist-test-slave-5l3k krb5kdc[14804](info): set up 2 sockets
Aug 06 06:46:38 dist-test-slave-5l3k krb5kdc[14804](info): commencing operation
krb5kdc: starting...
W20260806 06:46:39.581331 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:39 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:39.699120 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34261
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:34261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:39.781023 14817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:39.781215 14817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:39.781239 14817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:39.782325 14817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:39.782361 14817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:39.782375 14817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:39.782387 14817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:39.782399 14817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:39.783802 14817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:34261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:39.783998 14817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:39.784157 14817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:39.788985 14823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:39.789016 14824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:39.788960 14826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:39.789127 14817 server_base.cc:1034] running on GCE node
I20260806 06:46:39.789436 14817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:39.789686 14817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:39.790820 14817 hybrid_clock.cc:648] HybridClock initialized: now 1785998799790808 us; error 28 us; skew 500 ppm
Aug 06 06:46:39 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998799, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:39.793169 14817 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:39.793622 14817 webserver.cc:460] Webserver started at http://127.6.57.126:34209/ using document root <none> and password file <none>
I20260806 06:46:39.793802 14817 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:39.793840 14817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:39.793926 14817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:39.794971 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "9d45650079ad4a6cabc73f40fda202ee"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "0003c4cc38476c05b736bac237efc2ea"
server_key_iv: "2d75b4e26e384820631d43ade1c4b25e"
server_key_version: "encryptionkey@0"
I20260806 06:46:39.795266 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "9d45650079ad4a6cabc73f40fda202ee"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "0003c4cc38476c05b736bac237efc2ea"
server_key_iv: "2d75b4e26e384820631d43ade1c4b25e"
server_key_version: "encryptionkey@0"
I20260806 06:46:39.796576 14817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:39.797340 14833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:39.797560 14817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:39.797643 14817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "9d45650079ad4a6cabc73f40fda202ee"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "0003c4cc38476c05b736bac237efc2ea"
server_key_iv: "2d75b4e26e384820631d43ade1c4b25e"
server_key_version: "encryptionkey@0"
I20260806 06:46:39.797703 14817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:39.806684 14817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:39.808454 14817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:39.808585 14817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:39.812888 14817 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:34261
I20260806 06:46:39.812953 14885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:34261 every 8 connection(s)
I20260806 06:46:39.813297 14817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:39.813810 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14817
I20260806 06:46:39.813930 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:39.814147 6373 external_mini_cluster.cc:1428] Setting key 2a29eee6126d462f9d1c90e81dc5e8c0
I20260806 06:46:39.814320 14886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:39.817754 14886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Bootstrap starting.
I20260806 06:46:39.818423 14886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:39.818795 14886 log.cc:824] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:39.819664 14886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: No bootstrap required, opened a new log
Aug 06 06:46:39 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:39.821669 14886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:39.821857 14886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:39.821902 14886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d45650079ad4a6cabc73f40fda202ee, State: Initialized, Role: FOLLOWER
I20260806 06:46:39.822043 14886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:39.822145 14886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:39.822194 14886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:39.822243 14886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:39.822985 14886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:39.823101 14886 leader_election.cc:302] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d45650079ad4a6cabc73f40fda202ee; no voters:
I20260806 06:46:39.823283 14886 leader_election.cc:288] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:39.823365 14891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:39.823532 14886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:39.823551 14891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 LEADER]: Becoming Leader. State: Replica: 9d45650079ad4a6cabc73f40fda202ee, State: Running, Role: LEADER
I20260806 06:46:39.823774 14891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:39.824162 14891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d45650079ad4a6cabc73f40fda202ee. Latest consensus state: current_term: 1 leader_uuid: "9d45650079ad4a6cabc73f40fda202ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } } }
I20260806 06:46:39.824436 14891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:39.824342 14893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d45650079ad4a6cabc73f40fda202ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } } }
I20260806 06:46:39.824904 14893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:39.825068 14900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:39.825850 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:39.814724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.6.57.126:34261)
0806 06:46:39.814938 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:39.814947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:39.814981 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:46:39.815528 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:39.815559 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:39.815658 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:39.815811 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:39.816417 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:39.817765 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:39.818694 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:39.819226 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:39.819357 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:39.821795 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:39.821799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:39.821799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:39.821811 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:39.823854 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:39.824280 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:39.824283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:39.824283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:39.824414 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:39.824659 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:39.824660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:39.824660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:39.824994 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:39.825529 (+ 535us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:39.825589 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:46:39.825711 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":99,"threads_started":1}
I20260806 06:46:39.825901 14900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:39.827958 14900 catalog_manager.cc:1347] Generated new cluster ID: a6b971db8d76457dabb78ecb81516c52
I20260806 06:46:39.828028 14900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:39.859423 14900 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:39.860242 14900 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:39.868889 14900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Generated new TSK 0
I20260806 06:46:39.869120 14900 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:39.898998 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:39.981144 14914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:39.981356 14914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:39.981392 14914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:39.982684 14914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:39.982728 14914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:39.982766 14914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:39.984413 14914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:39.984664 14914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:39.984947 14914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:39.990293 14921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:39.990331 14924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:39.990332 14920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:39.990453 14914 server_base.cc:1034] running on GCE node
I20260806 06:46:39.990764 14914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:39.991029 14914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:39.992204 14914 hybrid_clock.cc:648] HybridClock initialized: now 1785998799992182 us; error 32 us; skew 500 ppm
Aug 06 06:46:39 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998799, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:39.994892 14914 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:39.995381 14914 webserver.cc:460] Webserver started at http://127.6.57.65:44951/ using document root <none> and password file <none>
I20260806 06:46:39.995652 14914 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:39.995730 14914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:39.995890 14914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:39.996984 14914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "267f115195c24a0ebf5d459e4d19d3c9"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "41d8223d3f360f2acfb68d58d74ac994"
server_key_iv: "fae5885a537c3553639e8d3318d1a318"
server_key_version: "encryptionkey@0"
I20260806 06:46:39.997377 14914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "267f115195c24a0ebf5d459e4d19d3c9"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "41d8223d3f360f2acfb68d58d74ac994"
server_key_iv: "fae5885a537c3553639e8d3318d1a318"
server_key_version: "encryptionkey@0"
I20260806 06:46:39.998916 14914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:39.999676 14931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:39.999850 14914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:39.999965 14914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "267f115195c24a0ebf5d459e4d19d3c9"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "41d8223d3f360f2acfb68d58d74ac994"
server_key_iv: "fae5885a537c3553639e8d3318d1a318"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.000047 14914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.017410 14914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.019901 14914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.020080 14914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.020326 14914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.020658 14914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.020716 14914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.020759 14914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.020790 14914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.027558 14914 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:37889
I20260806 06:46:40.027594 15047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:37889 every 8 connection(s)
I20260806 06:46:40.027966 14914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998799, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:40.034317 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 14914
I20260806 06:46:40.034447 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:40.034711 6373 external_mini_cluster.cc:1428] Setting key 6bf20817151c2500e59ca772fd60e3be
I20260806 06:46:40.038044 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.028688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:38177 (local address 127.6.57.126:34261)
0806 06:46:40.028735 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.028737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.028961 (+ 224us) server_negotiation.cc:407] Connection header received
0806 06:46:40.029430 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.029431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.029441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.029492 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.030445 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.030992 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.032033 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.032230 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.032280 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.034599 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:40.034601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:40.034601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:40.034609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:40.036445 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.036915 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.036916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.036917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.036987 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.037284 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.037285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.037285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.037427 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:40.037519 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:40.037832 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 06:46:40.037939 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:46:40.038255 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.028684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:38177)
0806 06:46:40.028842 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.028878 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.029349 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.029551 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.029554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.030161 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.030363 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.030368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.031084 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.031086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.031959 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.031961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.032306 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.032307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.032413 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.032981 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.032990 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.034443 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:40.036571 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.036579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.036581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.036827 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.037074 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.037075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.037075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.037206 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.037502 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:40.037510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:40.037673 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 06:46:40.037808 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 06:46:40.037979 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260806 06:46:40.038796 15048 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34261
I20260806 06:46:40.038911 15048 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:40.039115 15048 heartbeater.cc:511] Master 127.6.57.126:34261 requested a full tablet report, sending...
I20260806 06:46:40.039765 14850 ts_manager.cc:194] Registered new tserver with Master: 267f115195c24a0ebf5d459e4d19d3c9 (127.6.57.65:37889)
I20260806 06:46:40.040700 14850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:38177
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:40.059332 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:40.140496 15055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.140709 15055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.140749 15055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.141870 15055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.141933 15055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.141976 15055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:40.143421 15055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.143672 15055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.143900 15055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.148805 15063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.148813 15061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.148804 15060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.149072 15055 server_base.cc:1034] running on GCE node
I20260806 06:46:40.149286 15055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.149466 15055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.150592 15055 hybrid_clock.cc:648] HybridClock initialized: now 1785998800150568 us; error 32 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.153168 15055 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:40.153637 15055 webserver.cc:460] Webserver started at http://127.6.57.66:36993/ using document root <none> and password file <none>
I20260806 06:46:40.153861 15055 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.153904 15055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.154052 15055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:40.155157 15055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "cdc297a6a6704e0c9c63026ac5156799"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "e594510d5f4d39f8a8ca708a9fdb35a8"
server_key_iv: "ff9a6ac3118331ab724682e7ad12eeb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.155503 15055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "cdc297a6a6704e0c9c63026ac5156799"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "e594510d5f4d39f8a8ca708a9fdb35a8"
server_key_iv: "ff9a6ac3118331ab724682e7ad12eeb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.156766 15055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:40.157485 15070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.157650 15055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:40.157769 15055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "cdc297a6a6704e0c9c63026ac5156799"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "e594510d5f4d39f8a8ca708a9fdb35a8"
server_key_iv: "ff9a6ac3118331ab724682e7ad12eeb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.157841 15055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.184095 15055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.186367 15055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.186570 15055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.186800 15055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.187119 15055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.187180 15055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.187215 15055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.187242 15055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.192752 15055 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:45413
I20260806 06:46:40.192842 15185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:45413 every 8 connection(s)
I20260806 06:46:40.193123 15055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:40.194027 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15055
I20260806 06:46:40.194155 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:40.194371 6373 external_mini_cluster.cc:1428] Setting key cfbe7b27756713d282e05aa0b5f11f82
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:40.204098 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.193815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:52729 (local address 127.6.57.126:34261)
0806 06:46:40.193876 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.193878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.194058 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:46:40.194708 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.194710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.194722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.194798 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.195952 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.196711 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.198223 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.198550 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.198624 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.200955 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:40.200956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:40.200957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:40.200966 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:40.202634 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.203014 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.203017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.203017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.203086 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.203415 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.203417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.203417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.203548 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:40.203604 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:40.203965 (+ 361us) server_negotiation.cc:299] Negotiation successful
0806 06:46:40.204036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:46:40.204420 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.193769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:52729)
0806 06:46:40.193932 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.193969 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.194588 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.195080 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.195085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.195687 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.195866 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.195870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.196854 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.196855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.198124 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.198127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.198646 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.198647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.198746 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.199291 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.199298 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.200834 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:40.202712 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.202719 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.202720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.202934 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.203188 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.203189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.203190 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.203332 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.203621 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:40.203627 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:40.203803 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 06:46:40.203931 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 06:46:40.204121 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
I20260806 06:46:40.205016 15186 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34261
I20260806 06:46:40.205122 15186 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:40.205291 15186 heartbeater.cc:511] Master 127.6.57.126:34261 requested a full tablet report, sending...
I20260806 06:46:40.205711 14850 ts_manager.cc:194] Registered new tserver with Master: cdc297a6a6704e0c9c63026ac5156799 (127.6.57.66:45413)
I20260806 06:46:40.206041 14850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:52729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:40.221866 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:40.302335 15191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.302591 15191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.302640 15191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.303802 15191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.303856 15191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.303900 15191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:40.305306 15191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.305548 15191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.305802 15191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.310703 15199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.310704 15197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.310738 15191 server_base.cc:1034] running on GCE node
W20260806 06:46:40.310736 15196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.311174 15191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.311414 15191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.312568 15191 hybrid_clock.cc:648] HybridClock initialized: now 1785998800312550 us; error 40 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.315012 15191 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:40.315454 15191 webserver.cc:460] Webserver started at http://127.6.57.67:43153/ using document root <none> and password file <none>
I20260806 06:46:40.315711 15191 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.315778 15191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.315927 15191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:40.316915 15191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "ac98eec7d43d4fa5bfd197d56c354a04"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "d9109ed632344add1b3d83dbae16da24"
server_key_iv: "86f36da30c3f7d5c0cb70611879f7a56"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.317272 15191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "ac98eec7d43d4fa5bfd197d56c354a04"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "d9109ed632344add1b3d83dbae16da24"
server_key_iv: "86f36da30c3f7d5c0cb70611879f7a56"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.318629 15191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:46:40.319363 15206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.319515 15191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:40.319607 15191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "ac98eec7d43d4fa5bfd197d56c354a04"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "d9109ed632344add1b3d83dbae16da24"
server_key_iv: "86f36da30c3f7d5c0cb70611879f7a56"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.319693 15191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.329672 15191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.331521 15191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.331669 15191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.331866 15191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.332157 15191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.332211 15191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.332247 15191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.332273 15191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.338117 15191 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:45147
I20260806 06:46:40.338174 15322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:45147 every 8 connection(s)
I20260806 06:46:40.338446 15191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:40.347179 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15191
I20260806 06:46:40.347288 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:40.347460 6373 external_mini_cluster.cc:1428] Setting key f33ab4fc181e60f73117a9f1843cf00e
I20260806 06:46:40.347879 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.339253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:41737 (local address 127.6.57.126:34261)
0806 06:46:40.339319 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.339321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.339519 (+ 198us) server_negotiation.cc:407] Connection header received
0806 06:46:40.340025 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.340026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.340039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.340089 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.340945 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.341468 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.342432 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.342696 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.342781 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.344690 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:40.344691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:40.344692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:40.344699 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:40.346142 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.346686 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.346687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.346688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.346762 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.347055 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.347056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.347056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.347177 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:40.347258 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:40.347676 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 06:46:40.347780 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:46:40.348093 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.339230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:41737)
0806 06:46:40.339409 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.339442 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.339917 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.340155 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.340159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.340672 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.340840 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.340844 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.341563 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.341564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.342328 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.342330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.342794 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.342795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.342905 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.343327 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.343335 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.344586 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:40.346266 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.346273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.346276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.346584 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.346851 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:40.346852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:40.346853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:40.346971 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:40.347249 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:40.347256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:40.347423 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 06:46:40.347623 (+ 200us) client_negotiation.cc:240] Negotiation successful
0806 06:46:40.347806 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260806 06:46:40.348640 15323 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34261
I20260806 06:46:40.348718 15323 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:40.348850 15323 heartbeater.cc:511] Master 127.6.57.126:34261 requested a full tablet report, sending...
I20260806 06:46:40.349206 14850 ts_manager.cc:194] Registered new tserver with Master: ac98eec7d43d4fa5bfd197d56c354a04 (127.6.57.67:45147)
I20260806 06:46:40.349583 14850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:41737
I20260806 06:46:40.359469 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
Entry for principal oryx/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab.
WARNING: no policy specified for oryx/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
Entry for principal oryx/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab.
WARNING: no policy specified for oryx/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
Entry for principal oryx/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab.
I20260806 06:46:40.429523 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14914
I20260806 06:46:40.435038 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15055
I20260806 06:46:40.440802 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15191
I20260806 06:46:40.445477 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 14817
I20260806 06:46:40.451751 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34261
--webserver_interface=127.6.57.126
--webserver_port=34209
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:34261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:40.534624 15332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.534813 15332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.534834 15332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.535931 15332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:40.535967 15332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.535982 15332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.535995 15332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:40.536006 15332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:40.537433 15332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:34261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=34209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.537618 15332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.537817 15332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.542520 15340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.542580 15337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.542591 15338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.542753 15332 server_base.cc:1034] running on GCE node
I20260806 06:46:40.543020 15332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.543278 15332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.544421 15332 hybrid_clock.cc:648] HybridClock initialized: now 1785998800544403 us; error 33 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.547173 15332 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:40.547736 15332 webserver.cc:460] Webserver started at http://127.6.57.126:34209/ using document root <none> and password file <none>
I20260806 06:46:40.547938 15332 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.547991 15332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.549362 15332 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:40.549995 15347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.550211 15332 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:40.550304 15332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "9d45650079ad4a6cabc73f40fda202ee"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "0003c4cc38476c05b736bac237efc2ea"
server_key_iv: "2d75b4e26e384820631d43ade1c4b25e"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.550679 15332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.560604 15332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.562453 15332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.562671 15332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.566469 15332 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:34261
I20260806 06:46:40.566610 15402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:34261 every 8 connection(s)
I20260806 06:46:40.566932 15332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:40.568339 15403 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:46:40.569010 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Bootstrap starting.
I20260806 06:46:40.576565 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15332
I20260806 06:46:40.576884 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:37889
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:34261
--webserver_port=44951
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:46:40.577675 15403 log.cc:824] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:40.579958 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:46:40.580307 15403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Bootstrap complete.
I20260806 06:46:40.584303 15403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:40.584699 15403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d45650079ad4a6cabc73f40fda202ee, State: Initialized, Role: FOLLOWER
I20260806 06:46:40.584887 15403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:40.584988 15403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:40.585067 15403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:40.585131 15403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:46:40.586732 15403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:40.586881 15403 leader_election.cc:302] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d45650079ad4a6cabc73f40fda202ee; no voters:
I20260806 06:46:40.587097 15403 leader_election.cc:288] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:46:40.587215 15407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:46:40.587387 15407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [term 2 LEADER]: Becoming Leader. State: Replica: 9d45650079ad4a6cabc73f40fda202ee, State: Running, Role: LEADER
I20260806 06:46:40.587441 15403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:40.587517 15407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } }
I20260806 06:46:40.587904 15408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d45650079ad4a6cabc73f40fda202ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } } }
I20260806 06:46:40.587985 15408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:40.588284 15409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d45650079ad4a6cabc73f40fda202ee. Latest consensus state: current_term: 2 leader_uuid: "9d45650079ad4a6cabc73f40fda202ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d45650079ad4a6cabc73f40fda202ee" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34261 } } }
I20260806 06:46:40.588353 15409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:40.588521 15413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:40.589041 15413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:40.589231 15413 catalog_manager.cc:1259] Loaded cluster ID: a6b971db8d76457dabb78ecb81516c52
I20260806 06:46:40.589497 15413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:40.590971 15413 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:40.591167 15413 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d45650079ad4a6cabc73f40fda202ee: Loaded TSK: 0
I20260806 06:46:40.591379 15413 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:46:40.663540 15406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.663729 15406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.663751 15406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.664914 15406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.664955 15406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.664981 15406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:40.666388 15406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.65.keytab
--principal=oryx/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:37889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=44951
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.666651 15406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.666880 15406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.671803 15430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.671876 15431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.671849 15433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.672396 15406 server_base.cc:1034] running on GCE node
I20260806 06:46:40.672624 15406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.672904 15406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.674103 15406 hybrid_clock.cc:648] HybridClock initialized: now 1785998800674090 us; error 35 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.676659 15406 init.cc:377] Logged in from keytab as oryx/127.6.57.65@KRBTEST.COM (short username oryx)
I20260806 06:46:40.677136 15406 webserver.cc:460] Webserver started at http://127.6.57.65:44951/ using document root <none> and password file <none>
I20260806 06:46:40.677366 15406 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.677449 15406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.678889 15406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:40.679557 15440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.679772 15406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:40.679883 15406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "267f115195c24a0ebf5d459e4d19d3c9"
format_stamp: "Formatted at 2026-08-06 06:46:39 on dist-test-slave-5l3k"
server_key: "41d8223d3f360f2acfb68d58d74ac994"
server_key_iv: "fae5885a537c3553639e8d3318d1a318"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.680313 15406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.719862 15406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.721935 15406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.722046 15406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.722251 15406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.722518 15406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.722548 15406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.722571 15406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.722584 15406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.728415 15406 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:37889
I20260806 06:46:40.728549 15558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:37889 every 8 connection(s)
I20260806 06:46:40.728843 15406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:40.733714 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15406
I20260806 06:46:40.734027 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:45413
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:34261
--webserver_port=36993
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.737437 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.729574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:44733)
0806 06:46:40.729766 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.729807 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.730415 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.730872 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.730876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.731421 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.731605 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.731609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.732633 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.732633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.733452 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.733455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.734300 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.734301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.734404 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.734989 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.734997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.737291 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":129,"threads_started":1}
W20260806 06:46:40.737633 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.729678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:44733 (local address 127.6.57.126:34261)
0806 06:46:40.729852 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.729867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.729893 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:46:40.730636 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.730642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.730682 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.730792 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.731700 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.732542 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.733563 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.734210 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.734333 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.737423 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:44733: BlockingRecv error: recv got EOF from 127.6.57.65:44733 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
W20260806 06:46:40.737709 15559 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:34261 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.57.126:34261: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.742396 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.737980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:50055)
0806 06:46:40.738055 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.738065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.738100 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.738240 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.738242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.738396 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.738451 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.738452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.739100 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.739101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.739683 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.739686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.740045 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.740047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.740120 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.740574 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.740578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.742302 (+ 1724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260806 06:46:40.742424 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.738043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:50055 (local address 127.6.57.126:34261)
0806 06:46:40.738119 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.738121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.738144 (+ 23us) server_negotiation.cc:407] Connection header received
0806 06:46:40.738163 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.738164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.738173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.738214 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.738559 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.739005 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.739771 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.739961 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.740045 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.742383 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:50055: BlockingRecv error: recv got EOF from 127.6.57.65:50055 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.747188 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.742668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:60273)
0806 06:46:40.742803 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.742813 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.742851 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.742989 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.742990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.743152 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.743188 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.743189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.743753 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.743754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.744594 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.744597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.744902 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.744903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.744961 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.745333 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.745339 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.747106 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260806 06:46:40.747246 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.742786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:60273 (local address 127.6.57.126:34261)
0806 06:46:40.742833 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.742835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.742843 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:40.742910 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.742910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.742919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.742966 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.743262 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.743680 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.744674 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.744833 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.744879 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.747185 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:60273: BlockingRecv error: recv got EOF from 127.6.57.65:60273 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 06:46:40.747310 15559 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:46:40.818821 15563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.819054 15563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.819089 15563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.820186 15563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.820250 15563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.820290 15563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:40.821717 15563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.66.keytab
--principal=oryx/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:45413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=36993
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.821979 15563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.822221 15563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.827105 15569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.827147 15570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.827118 15572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.827273 15563 server_base.cc:1034] running on GCE node
I20260806 06:46:40.827490 15563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.827729 15563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.828894 15563 hybrid_clock.cc:648] HybridClock initialized: now 1785998800828886 us; error 51 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.831360 15563 init.cc:377] Logged in from keytab as oryx/127.6.57.66@KRBTEST.COM (short username oryx)
I20260806 06:46:40.831712 15563 webserver.cc:460] Webserver started at http://127.6.57.66:36993/ using document root <none> and password file <none>
I20260806 06:46:40.831871 15563 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.831938 15563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.833184 15563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:40.833719 15579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.833866 15563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:40.833949 15563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "cdc297a6a6704e0c9c63026ac5156799"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "e594510d5f4d39f8a8ca708a9fdb35a8"
server_key_iv: "ff9a6ac3118331ab724682e7ad12eeb5"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.834223 15563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.843614 15563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.845330 15563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.845436 15563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.845611 15563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.845873 15563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.845903 15563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.845923 15563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.845935 15563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.853204 15563 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:45413
I20260806 06:46:40.853267 15694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:45413 every 8 connection(s)
I20260806 06:46:40.853664 15563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.861042 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15563
W20260806 06:46:40.861316 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.854536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:46325 (local address 127.6.57.126:34261)
0806 06:46:40.854583 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.854585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.854917 (+ 332us) server_negotiation.cc:407] Connection header received
0806 06:46:40.855543 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.855545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.855563 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.855641 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.856817 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.857326 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.858299 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.858522 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.858581 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.861260 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:46325: BlockingRecv error: recv got EOF from 127.6.57.66:46325 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:46:40.861330 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:45147
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:34261
--webserver_port=43153
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:46:40.861351 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.854569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:46325)
0806 06:46:40.854783 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.854840 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.855435 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.855738 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.855743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.856532 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.856720 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.856725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.857405 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.857406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.858223 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.858225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.858602 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.858603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.858701 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.859095 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.859102 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.861152 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
W20260806 06:46:40.861526 15695 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:34261 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.57.126:34261: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.866400 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.861767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:55529)
0806 06:46:40.861807 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.861817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.861854 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.861966 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.861967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.862350 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.862399 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.862402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.863279 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.863280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.863744 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.863745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.864214 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.864215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.864260 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.864692 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.864697 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.866357 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:46:40.866600 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.861796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:55529 (local address 127.6.57.126:34261)
0806 06:46:40.861837 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.861839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.861847 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:40.861905 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.861906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.861915 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.861964 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.862558 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.863190 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.863827 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.864131 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.864218 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.866508 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:55529: BlockingRecv error: recv got EOF from 127.6.57.66:55529 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.871630 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.866945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:56603)
0806 06:46:40.867084 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.867098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.867159 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.867302 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.867303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.867466 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.867501 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.867502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.868073 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.868074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.868908 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.868911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.869277 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.869279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.869343 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.869710 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.869716 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.871550 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122}
W20260806 06:46:40.871683 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.867032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:56603 (local address 127.6.57.126:34261)
0806 06:46:40.867095 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.867097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.867142 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:46:40.867225 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.867226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.867233 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.867278 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.867576 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.867993 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.869015 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.869199 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.869251 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.871621 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:56603: BlockingRecv error: recv got EOF from 127.6.57.66:56603 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 06:46:40.872051 15695 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:46:40.947239 15698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:40.947425 15698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:40.947448 15698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:40.948588 15698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:40.948628 15698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:40.948679 15698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:40.950146 15698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/krb5kdc/oryx_127.6.57.67.keytab
--principal=oryx/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:45147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=43153
--tserver_master_addrs=127.6.57.126:34261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:40.950420 15698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:40.950665 15698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:40.955565 15704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:40.955572 15703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.955667 15698 server_base.cc:1034] running on GCE node
W20260806 06:46:40.955614 15706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:40.955948 15698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:40.956205 15698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:40.957371 15698 hybrid_clock.cc:648] HybridClock initialized: now 1785998800957343 us; error 39 us; skew 500 ppm
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998800, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:40.959976 15698 init.cc:377] Logged in from keytab as oryx/127.6.57.67@KRBTEST.COM (short username oryx)
I20260806 06:46:40.960424 15698 webserver.cc:460] Webserver started at http://127.6.57.67:43153/ using document root <none> and password file <none>
I20260806 06:46:40.960665 15698 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:40.960741 15698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:40.962075 15698 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:40.962747 15713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.962932 15698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:40.963023 15698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "ac98eec7d43d4fa5bfd197d56c354a04"
format_stamp: "Formatted at 2026-08-06 06:46:40 on dist-test-slave-5l3k"
server_key: "d9109ed632344add1b3d83dbae16da24"
server_key_iv: "86f36da30c3f7d5c0cb70611879f7a56"
server_key_version: "encryptionkey@0"
I20260806 06:46:40.963332 15698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:40.972391 15698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:40.974738 15698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:40.974891 15698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:40.975106 15698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:40.975399 15698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:40.975433 15698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.975453 15698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:40.975466 15698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:40.981518 15698 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:45147
I20260806 06:46:40.981562 15830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:45147 every 8 connection(s)
I20260806 06:46:40.981874 15698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:40.987443 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15698
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 06:46:40.989799 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.982504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:47177 (local address 127.6.57.126:34261)
0806 06:46:40.982549 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.982556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.982761 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:46:40.983239 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.983241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.983255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.983316 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.984348 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.984985 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.985967 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.986186 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.986259 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.989737 (+ 3478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:47177: BlockingRecv error: recv got EOF from 127.6.57.67:47177 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:46:40.989876 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.982462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:47177)
0806 06:46:40.982647 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.982679 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.983136 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.983385 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.983388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.984058 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.984243 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.984248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.985084 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.985087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.985870 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.985872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.986279 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.986280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.986394 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.986990 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.987017 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.989682 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
W20260806 06:46:40.990062 15831 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:34261 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.57.126:34261: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.994576 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.990262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:34401)
0806 06:46:40.990314 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.990323 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.990359 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.990508 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.990509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.990644 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.990689 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.990690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.991243 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.991244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.991879 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.991881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.992252 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.992253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.992318 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.992725 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.992730 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.994502 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:46:40.994670 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.990328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34401 (local address 127.6.57.126:34261)
0806 06:46:40.990360 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.990362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.990369 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:40.990411 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.990416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.990423 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.990468 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.990757 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.991161 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.991976 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.992175 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.992240 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.994620 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:34401: BlockingRecv error: recv got EOF from 127.6.57.67:34401 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:46:40.995579 15835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.988041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.6.57.126:34261)
0806 06:46:40.988188 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.988190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.988199 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:40.988265 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.988266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.988282 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.988350 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.989078 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.989600 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.990979 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.991177 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.991248 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.992343 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:40.992347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:40.992348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:40.992359 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:40.994194 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.994583 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.994586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.994586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.994696 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:40.994902 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:40.994903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:40.994903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:40.995181 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:40.995274 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:40.995389 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 06:46:40.995465 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:40 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:40.998942 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:40.994999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:60165)
0806 06:46:40.995047 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:40.995056 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:40.995088 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:40.995183 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:40.995183 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:40.995313 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:40.995344 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.995346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.996155 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.996156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.996755 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:40.996757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.997050 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.997051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:40.997085 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.997376 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:40.997379 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:40.998904 (+ 1525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:46:40.999068 15831 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:46:40.999178 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:40.995057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:60165 (local address 127.6.57.126:34261)
0806 06:46:40.995094 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:40.995096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:40.995104 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:40.995133 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:40.995134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:40.995141 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:40.995181 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:40.995423 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.996068 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.996820 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:40.996986 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:40.997037 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:40.998999 (+ 1962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:60165: BlockingRecv error: recv got EOF from 127.6.57.67:60165 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 06:46:41 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:41 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:41.756577 15836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:41.747704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:46021)
0806 06:46:41.747848 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:41.747860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:41.747922 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:41.748066 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:41.748066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:41.748235 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:41.748285 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:41.748286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.748912 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.748913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:41.749436 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:41.749438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.749757 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.749758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:41.749802 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:41.754661 (+ 4859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:41.754667 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:41.756503 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
W20260806 06:46:41.756645 15837 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:41.747781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:46021 (local address 127.6.57.126:34261)
0806 06:46:41.747917 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:41.747918 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:41.747928 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:41.747972 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:41.747972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:41.747982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:41.748047 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:41.748370 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.748845 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.749504 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.749690 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.749741 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:41.756585 (+ 6844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:46021: BlockingRecv error: recv got EOF from 127.6.57.65:46021 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
Aug 06 06:46:41 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:41 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:41.877236 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:41.872455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:49975)
0806 06:46:41.872634 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:41.872647 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:41.872737 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:41.872903 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:41.872904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:41.873107 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:41.873156 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:41.873158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.873842 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.873843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:41.874379 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:41.874381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.874911 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.874912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:41.874974 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:41.875401 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:41.875407 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:41.877172 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
W20260806 06:46:41.877329 15837 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:41.872534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:49975 (local address 127.6.57.126:34261)
0806 06:46:41.872618 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:41.872621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:41.872719 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:46:41.872795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:41.872796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:41.872810 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:41.872881 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:41.873243 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.873746 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.874466 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:41.874834 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:41.874913 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:41.877263 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:49975: BlockingRecv error: recv got EOF from 127.6.57.66:49975 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:42.004415 15839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:41.999442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:48303)
0806 06:46:41.999578 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:41.999591 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:41.999683 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:41.999865 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:41.999866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:42.000047 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:42.000097 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.000099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.000914 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.000915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.001580 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.001582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.001942 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.001943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.001996 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.002515 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:42.002520 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:42.004307 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
W20260806 06:46:42.004455 15837 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:41.999528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:48303 (local address 127.6.57.126:34261)
0806 06:46:41.999582 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:41.999584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:41.999663 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:46:41.999761 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:41.999762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:41.999777 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:41.999855 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:42.000200 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.000829 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.001668 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.001858 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.001943 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.004387 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:48303: BlockingRecv error: recv got EOF from 127.6.57.67:48303 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:42.761724 15840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:42.757133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:34675)
0806 06:46:42.757283 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:42.757294 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:42.757357 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:42.757513 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:42.757513 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:42.757671 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:42.757719 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.757721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.758383 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.758383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.758975 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.758977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.759286 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.759287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.759325 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.759862 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:42.759868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:42.761662 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
W20260806 06:46:42.761813 15841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:42.757208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:34675 (local address 127.6.57.126:34261)
0806 06:46:42.757314 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:42.757316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:42.757340 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:46:42.757414 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:42.757415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:42.757426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:42.757498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:42.757810 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.758318 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.759041 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.759224 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.759273 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.761744 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:34675: BlockingRecv error: recv got EOF from 127.6.57.65:34675 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:42 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:42.882342 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:42.877676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:45177)
0806 06:46:42.877835 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:42.877847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:42.877919 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:42.878053 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:42.878054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:42.878204 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:42.878253 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.878254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.878933 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.878935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.879474 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:42.879476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.879964 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.879965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:42.880031 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.880526 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:42.880531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:42.882281 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
W20260806 06:46:42.882434 15841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:42.877790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:45177 (local address 127.6.57.126:34261)
0806 06:46:42.877861 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:42.877863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:42.877915 (+ 52us) server_negotiation.cc:407] Connection header received
0806 06:46:42.877973 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:42.877974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:42.877985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:42.878038 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:42.878310 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.878867 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.879561 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:42.879887 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:42.879954 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:42.882368 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:45177: BlockingRecv error: recv got EOF from 127.6.57.66:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:43.010165 15843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:43.005100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:59395)
0806 06:46:43.005240 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:43.005251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:43.005316 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:43.005500 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:43.005501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:43.005684 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:43.005732 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.005734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.006426 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.006427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.006989 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.006991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.007364 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.007365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.007433 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.007887 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:43.007891 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:43.010025 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
W20260806 06:46:43.010214 15841 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:43.005273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:59395 (local address 127.6.57.126:34261)
0806 06:46:43.005351 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:43.005354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:43.005363 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:43.005387 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:43.005388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:43.005407 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:43.005486 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:43.005817 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.006342 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.007084 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.007287 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.007365 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.010146 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:59395: BlockingRecv error: recv got EOF from 127.6.57.67:59395 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:43.767413 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:43.762252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:35551)
0806 06:46:43.762442 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:43.762455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:43.762584 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:43.762728 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:43.762730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:43.762898 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:43.762947 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.762948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.763599 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.763601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.764209 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.764211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.764574 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.764575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.764616 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.765185 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:43.765190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:43.767302 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
W20260806 06:46:43.767457 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:43.762392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:35551 (local address 127.6.57.126:34261)
0806 06:46:43.762559 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:43.762566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:43.762576 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:43.762634 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:43.762635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:43.762645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:43.762705 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:43.763009 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.763507 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.764290 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.764501 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.764569 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.767382 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:35551: BlockingRecv error: recv got EOF from 127.6.57.65:35551 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:43 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:43.887699 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:43.882815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:58061)
0806 06:46:43.882983 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:43.882994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:43.883082 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:43.883241 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:43.883242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:43.883429 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:43.883477 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.883478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.884136 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.884138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.884680 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:43.884682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.885134 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.885135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:43.885196 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.885733 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:43.885737 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:43.887606 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
W20260806 06:46:43.887770 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:43.882896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:58061 (local address 127.6.57.126:34261)
0806 06:46:43.882969 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:43.882973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:43.883062 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:46:43.883170 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:43.883172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:43.883184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:43.883239 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:43.883552 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.884049 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.884763 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:43.885038 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:43.885135 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:43.887709 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:58061: BlockingRecv error: recv got EOF from 127.6.57.66:58061 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:44.015673 15847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:44.010791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:55837)
0806 06:46:44.010924 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:44.010964 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:44.011021 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:44.011177 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:44.011179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:44.011359 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:44.011404 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.011405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.012071 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.012073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.012604 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.012606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.012967 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.012968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.013027 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.013470 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:44.013474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:44.015565 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
W20260806 06:46:44.015724 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:44.010913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:55837 (local address 127.6.57.126:34261)
0806 06:46:44.011004 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:44.011006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:44.011016 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:44.011077 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:44.011078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:44.011091 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:44.011166 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:44.011486 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.011975 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.012704 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.012886 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.012955 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.015646 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:55837: BlockingRecv error: recv got EOF from 127.6.57.67:55837 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:44.772804 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:44.768090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:54617)
0806 06:46:44.768228 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:44.768241 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:44.768296 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:44.768450 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:44.768451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:44.768604 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:44.768658 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.768660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.769314 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.769314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.769819 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.769820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.770170 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.770171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.770210 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.770691 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:44.770696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:44.772707 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260806 06:46:44.772858 15849 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:44.768164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54617 (local address 127.6.57.126:34261)
0806 06:46:44.768322 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:44.768324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:44.768332 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:44.768357 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:44.768358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:44.768369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:44.768436 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:44.768727 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.769245 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.769887 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.770106 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.770160 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.772786 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:54617: BlockingRecv error: recv got EOF from 127.6.57.65:54617 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:44 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:44.892685 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:44.888163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:57615)
0806 06:46:44.888337 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:44.888347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:44.888409 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:44.888578 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:44.888580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:44.888830 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:44.888922 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.888924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.889604 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.889605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.890109 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:44.890110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.890439 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.890440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:44.890515 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.890982 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:44.890987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:44.892621 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
W20260806 06:46:44.892781 15849 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:44.888215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:57615 (local address 127.6.57.126:34261)
0806 06:46:44.888284 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:44.888286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:44.888416 (+ 130us) server_negotiation.cc:407] Connection header received
0806 06:46:44.888458 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:44.888458 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:44.888487 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:44.888549 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:44.888990 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.889539 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.890185 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:44.890376 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:44.890429 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:44.892711 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:57615: BlockingRecv error: recv got EOF from 127.6.57.66:57615 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:45.021131 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:45.016309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:47297)
0806 06:46:45.016446 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:45.016460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:45.016531 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:45.016682 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:45.016683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:45.016846 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:45.016896 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.016897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.017536 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.017537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.018163 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.018166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.018616 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.018618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.018686 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.019188 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:45.019194 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:45.021073 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":57,"threads_started":1}
W20260806 06:46:45.021247 15849 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:45.016415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:47297 (local address 127.6.57.126:34261)
0806 06:46:45.016507 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:45.016510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:45.016520 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:45.016594 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:45.016595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:45.016607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:45.016660 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:45.016958 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.017461 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.018270 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.018543 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.018609 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.021166 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:47297: BlockingRecv error: recv got EOF from 127.6.57.67:47297 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:45.778318 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:45.773312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:59941)
0806 06:46:45.773474 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:45.773488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:45.773554 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:45.773686 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:45.773687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:45.773848 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:45.773898 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.773899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.774783 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.774784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.775311 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.775313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.775639 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.775640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.775678 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.776232 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:45.776239 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:45.778217 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
W20260806 06:46:45.778376 15853 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:45.773378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:59941 (local address 127.6.57.126:34261)
0806 06:46:45.773541 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:45.773543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:45.773551 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:45.773600 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:45.773601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:45.773612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:45.773667 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:45.773963 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.774704 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.775381 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.775571 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.775624 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.778300 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:59941: BlockingRecv error: recv got EOF from 127.6.57.65:59941 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:45 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:45.897564 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:45.893209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:37281)
0806 06:46:45.893381 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:45.893391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:45.893444 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:45.893579 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:45.893580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:45.893727 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:45.893771 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.893773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.894345 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.894345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.894896 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:45.894898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.895274 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.895275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:45.895312 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.895750 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:45.895754 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:45.897491 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
W20260806 06:46:45.897698 15853 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:45.893323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37281 (local address 127.6.57.126:34261)
0806 06:46:45.893393 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:45.893395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:45.893429 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:46:45.893497 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:45.893498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:45.893507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:45.893562 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:45.893824 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.894285 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.894975 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:45.895203 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:45.895268 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:45.897618 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:37281: BlockingRecv error: recv got EOF from 127.6.57.66:37281 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:46.026651 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:46.021667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:55897)
0806 06:46:46.021881 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:46.021893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:46.021965 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:46.022176 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:46.022177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:46.022368 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:46.022415 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.022416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.023119 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.023120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.023677 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.023678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.024070 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.024071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.024132 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.024572 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:46.024577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:46.026562 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":104,"threads_started":1}
W20260806 06:46:46.026680 15853 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:46.021763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:55897 (local address 127.6.57.126:34261)
0806 06:46:46.021867 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:46.021869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:46.021948 (+ 79us) server_negotiation.cc:407] Connection header received
0806 06:46:46.022077 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:46.022083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:46.022096 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:46.022167 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:46.022512 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.023023 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.023762 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.023968 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.024069 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.026620 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:55897: BlockingRecv error: recv got EOF from 127.6.57.67:55897 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:46.788216 15856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:46.778830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:35991)
0806 06:46:46.778977 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:46.778987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:46.779039 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:46.779166 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:46.779167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:46.779345 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:46.779395 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.779397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.780026 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.780026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.780542 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.780543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.780934 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.780935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.780973 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.786320 (+ 5347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:46.786326 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:46.788157 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
W20260806 06:46:46.788371 15859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:46.778918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:35991 (local address 127.6.57.126:34261)
0806 06:46:46.779061 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:46.779063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:46.779071 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:46.779093 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:46.779093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:46.779104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:46.779162 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:46.779465 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.779957 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.780645 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.780855 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.780928 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.788274 (+ 7346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:35991: BlockingRecv error: recv got EOF from 127.6.57.65:35991 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:46 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:46.902818 15860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:46.898107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:33857)
0806 06:46:46.898276 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:46.898287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:46.898348 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:46.898584 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:46.898585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:46.898756 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:46.898803 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.898804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.899503 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.899504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.899990 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:46.899991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.900349 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.900350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:46.900386 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.900857 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:46.900861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:46.902761 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
W20260806 06:46:46.903008 15859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:46.898210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:33857 (local address 127.6.57.126:34261)
0806 06:46:46.898299 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:46.898301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:46.898336 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:46:46.898425 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:46.898426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:46.898438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:46.898500 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:46.898900 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.899434 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.900093 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:46.900285 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:46.900343 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:46.902915 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:33857: BlockingRecv error: recv got EOF from 127.6.57.66:33857 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:47.032560 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:47.027173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:34239)
0806 06:46:47.027379 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:47.027394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:47.027471 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:47.027644 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:47.027646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:47.027833 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:47.027886 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.027888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.028597 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.028599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.029143 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.029144 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.029703 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.029705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.029779 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.030429 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:47.030434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:47.032491 (+ 2057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
W20260806 06:46:47.032648 15859 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:47.027295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34239 (local address 127.6.57.126:34261)
0806 06:46:47.027384 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:47.027387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:47.027449 (+ 62us) server_negotiation.cc:407] Connection header received
0806 06:46:47.027544 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:47.027545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:47.027567 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:47.027637 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:47.027970 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.028490 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.029346 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.029583 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.029681 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.032575 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:34239: BlockingRecv error: recv got EOF from 127.6.57.67:34239 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:47.794934 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:47.788634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:54725)
0806 06:46:47.788782 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:47.788797 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:47.788873 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:47.789097 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:47.789098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:47.789419 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:47.789505 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.789508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.790447 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.790448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.791452 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.791456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.791971 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.791973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.792039 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.792545 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:47.792552 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:47.794815 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
W20260806 06:46:47.794968 15863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:47.788806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54725 (local address 127.6.57.126:34261)
0806 06:46:47.788906 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:47.788908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:47.788917 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:47.788945 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:47.788946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:47.788959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:47.789027 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:47.789582 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.790367 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.791560 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.791899 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.791973 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.794893 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:54725: BlockingRecv error: recv got EOF from 127.6.57.65:54725 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:47 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:47.908005 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:47.903359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:33549)
0806 06:46:47.903548 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:47.903560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:47.903623 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:47.903759 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:47.903760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:47.903918 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:47.903967 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.903969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.904624 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.904625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.905158 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:47.905159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.905462 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.905463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:47.905499 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.905958 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:47.905963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:47.907950 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":107,"threads_started":1}
W20260806 06:46:47.908118 15863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:47.903434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:33549 (local address 127.6.57.126:34261)
0806 06:46:47.903509 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:47.903511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:47.903618 (+ 107us) server_negotiation.cc:407] Connection header received
0806 06:46:47.903679 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:47.903680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:47.903692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:47.903732 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:47.904037 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.904547 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.905225 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:47.905400 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:47.905448 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:47.908032 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:33549: BlockingRecv error: recv got EOF from 127.6.57.66:33549 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:48.038076 15865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:48.033145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:59701)
0806 06:46:48.033327 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:48.033338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:48.033426 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:48.033571 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:48.033572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:48.033755 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:48.033800 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.033802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.034528 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.034529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.035073 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.035075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.035445 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.035446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.035509 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.035930 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:48.035934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:48.037967 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":94,"threads_started":1}
W20260806 06:46:48.038108 15863 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:48.033237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:59701 (local address 127.6.57.126:34261)
0806 06:46:48.033320 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:48.033322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:48.033408 (+ 86us) server_negotiation.cc:407] Connection header received
0806 06:46:48.033480 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:48.033481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:48.033495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:48.033566 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:48.033904 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.034424 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.035167 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.035359 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.035443 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.038041 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:59701: BlockingRecv error: recv got EOF from 127.6.57.67:59701 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:48.800455 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:48.795688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:49015)
0806 06:46:48.795853 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:48.795869 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:48.795937 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:48.796127 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:48.796128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:48.796320 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:48.796372 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.796373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.797028 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.797029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.797565 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.797567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.797927 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.797927 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.797972 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.798594 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:48.798599 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:48.800398 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
W20260806 06:46:48.800580 15867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:48.795832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:49015 (local address 127.6.57.126:34261)
0806 06:46:48.795982 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:48.795984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:48.795991 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:48.796012 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:48.796013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:48.796022 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:48.796104 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:48.796440 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.796953 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.797646 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.797855 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.797909 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.800494 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:49015: BlockingRecv error: recv got EOF from 127.6.57.65:49015 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:48 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:48.913544 15868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:48.908541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:38943)
0806 06:46:48.908728 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:48.908752 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:48.908799 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:48.908959 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:48.908960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:48.909129 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:48.909176 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.909178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.909919 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.909919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.910453 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:48.910455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.910890 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.910891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:48.910948 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.911542 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:48.911548 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:48.913470 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
W20260806 06:46:48.913659 15867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:48.908772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:38943 (local address 127.6.57.126:34261)
0806 06:46:48.908840 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:48.908842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:48.908849 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:46:48.908874 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:48.908875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:48.908885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:48.908938 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:48.909243 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.909847 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.910612 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:48.910804 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:48.910857 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:48.913566 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:38943: BlockingRecv error: recv got EOF from 127.6.57.66:38943 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:49.043910 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:49.038607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:39075)
0806 06:46:49.038781 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:49.038791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:49.038847 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:49.039084 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:49.039085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:49.039307 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:49.039374 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.039376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.040405 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.040407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.041080 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.041084 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.041553 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.041554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.041604 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.042062 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:49.042069 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:49.043853 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
W20260806 06:46:49.044075 15867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:49.038681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:39075 (local address 127.6.57.126:34261)
0806 06:46:49.038740 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:49.038742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:49.038849 (+ 107us) server_negotiation.cc:407] Connection header received
0806 06:46:49.038941 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:49.038942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:49.038961 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:49.039027 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:49.039443 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.040329 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.041169 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.041480 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.041546 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.043935 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:39075: BlockingRecv error: recv got EOF from 127.6.57.67:39075 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:49.805896 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:49.801071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:53985)
0806 06:46:49.801251 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:49.801266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:49.801347 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:49.801510 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:49.801511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:49.801699 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:49.801754 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.801755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.802415 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.802416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.803023 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.803025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.803408 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.803409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.803448 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.803951 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:49.803957 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:49.805825 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
W20260806 06:46:49.805976 15871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:49.801183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:53985 (local address 127.6.57.126:34261)
0806 06:46:49.801332 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:49.801334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:49.801343 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:49.801399 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:49.801400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:49.801411 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:49.801479 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:49.801832 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.802341 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.803106 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.803321 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.803378 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.805921 (+ 2543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:53985: BlockingRecv error: recv got EOF from 127.6.57.65:53985 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:49 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:49.919159 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:49.914020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:42635)
0806 06:46:49.914192 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:49.914207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:49.914276 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:49.914425 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:49.914426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:49.914621 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:49.914673 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.914674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.915376 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.915377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.916085 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:49.916088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.916393 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.916394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:49.916429 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.916951 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:49.916956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:49.919058 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":57,"threads_started":1}
W20260806 06:46:49.919194 15871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:49.914232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:42635 (local address 127.6.57.126:34261)
0806 06:46:49.914299 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:49.914301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:49.914312 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:49.914340 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:49.914340 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:49.914351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:49.914401 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:49.914741 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.915288 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.916158 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:49.916333 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:49.916383 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:49.919137 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:42635: BlockingRecv error: recv got EOF from 127.6.57.66:42635 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:50.049176 15873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:50.044376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:59051)
0806 06:46:50.044522 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:50.044535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:50.044598 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:50.044822 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:50.044823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:50.045039 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:50.045086 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.045088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.045814 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.045816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.046333 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.046335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.046786 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.046788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.046843 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.047279 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:50.047284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:50.049037 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
W20260806 06:46:50.049196 15871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:50.044425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:59051 (local address 127.6.57.126:34261)
0806 06:46:50.044475 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:50.044477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:50.044579 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:46:50.044667 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:50.044668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:50.044681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:50.044774 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:50.045177 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.045727 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.046420 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.046712 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.046789 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.049133 (+ 2344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:59051: BlockingRecv error: recv got EOF from 127.6.57.67:59051 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:50.811100 15874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:50.806585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:59619)
0806 06:46:50.806758 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:50.806769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:50.806821 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:50.806954 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:50.806955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:50.807118 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:50.807168 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.807169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.807791 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.807792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.808297 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.808298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.808622 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.808623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.808661 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.809207 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:50.809211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:50.811043 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":94,"threads_started":1}
W20260806 06:46:50.811246 15875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:50.806675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:59619 (local address 127.6.57.126:34261)
0806 06:46:50.806826 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:50.806828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:50.806837 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:50.806868 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:50.806868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:50.806879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:50.806935 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:50.807231 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.807728 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.808366 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.808556 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.808609 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.811140 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:59619: BlockingRecv error: recv got EOF from 127.6.57.65:59619 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:50 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:50.924328 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:50.919702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:54523)
0806 06:46:50.919844 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:50.919857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:50.919927 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:50.920088 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:50.920089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:50.920337 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:50.920399 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.920402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.921035 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.921036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.921527 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:50.921528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.921842 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.921842 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:50.921879 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.922305 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:50.922309 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:50.924261 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260806 06:46:50.924459 15875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:50.919789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:54523 (local address 127.6.57.126:34261)
0806 06:46:50.919841 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:50.919843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:50.919930 (+ 87us) server_negotiation.cc:407] Connection header received
0806 06:46:50.919989 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:50.919989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:50.920004 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:50.920063 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:50.920481 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.920969 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.921602 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:50.921780 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:50.921828 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:50.924356 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:54523: BlockingRecv error: recv got EOF from 127.6.57.66:54523 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:51.054152 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:51.049685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:37859)
0806 06:46:51.049831 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:51.049841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:51.049891 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:51.050022 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:51.050023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:51.050180 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:51.050228 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.050229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.050920 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.050921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.051446 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.051447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.051771 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.051772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.051811 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.052248 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:51.052252 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:51.054094 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
W20260806 06:46:51.054246 15875 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:51.049786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:37859 (local address 127.6.57.126:34261)
0806 06:46:51.049831 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:51.049833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:51.049878 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:46:51.049943 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:51.049944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:51.049955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:51.049995 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:51.050288 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.050855 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.051513 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.051702 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.051751 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.054174 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:37859: BlockingRecv error: recv got EOF from 127.6.57.67:37859 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:51.820147 15878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:51.811570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:53107)
0806 06:46:51.811728 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:51.811739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:51.811797 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:51.811945 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:51.811946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:51.812114 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:51.812160 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.812161 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.812791 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.812792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.813304 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.813306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.813611 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.813612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.813655 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.818066 (+ 4411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:51.818072 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:51.820067 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
W20260806 06:46:51.820221 15879 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:51.811668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:53107 (local address 127.6.57.126:34261)
0806 06:46:51.811818 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:51.811820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:51.811828 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:46:51.811850 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:51.811850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:51.811861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:51.811921 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:51.812219 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.812726 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.813368 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.813548 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.813595 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.820166 (+ 6571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:53107: BlockingRecv error: recv got EOF from 127.6.57.65:53107 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:51 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:51.929824 15880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:51.924856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:36511)
0806 06:46:51.925015 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:51.925029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:51.925107 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:51.925293 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:51.925295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:51.925535 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:51.925587 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.925589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.926243 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.926244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.926902 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:51.926905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.927305 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.927307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:51.927357 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.927794 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:51.927799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:51.929738 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
W20260806 06:46:51.929878 15879 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:51.924880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:36511 (local address 127.6.57.126:34261)
0806 06:46:51.924942 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:51.924944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:51.925096 (+ 152us) server_negotiation.cc:407] Connection header received
0806 06:46:51.925186 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:51.925187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:51.925215 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:51.925269 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:51.925655 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.926163 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.927015 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:51.927211 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:51.927264 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:51.929829 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:36511: BlockingRecv error: recv got EOF from 127.6.57.66:36511 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:52.059563 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:52.054661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:36557)
0806 06:46:52.054838 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:52.054851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:52.054945 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:52.055124 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:52.055126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:52.055329 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:52.055397 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.055399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.056117 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.056118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.056665 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.056667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.057101 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.057103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.057165 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.057625 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:52.057630 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:52.059494 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
W20260806 06:46:52.059765 15879 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:52.054741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36557 (local address 127.6.57.126:34261)
0806 06:46:52.054818 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:52.054821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:52.054925 (+ 104us) server_negotiation.cc:407] Connection header received
0806 06:46:52.055014 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:52.055015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:52.055048 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:52.055111 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:52.055505 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.056014 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.056783 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.056982 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.057100 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.059677 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:36557: BlockingRecv error: recv got EOF from 127.6.57.67:36557 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:52.825304 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:52.820683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:35937)
0806 06:46:52.820809 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:52.820828 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:52.820877 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:52.821041 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:52.821042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:52.821215 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:52.821262 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.821264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.821912 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.821913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.822436 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.822437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.822870 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.822871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.822929 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.823350 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:52.823355 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:52.825228 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
W20260806 06:46:52.825366 15883 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:52.820739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:35937 (local address 127.6.57.126:34261)
0806 06:46:52.820901 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:52.820903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:52.820912 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:46:52.820943 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:52.820943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:52.820953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:52.821015 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:52.821332 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.821833 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.822602 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.822801 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.822857 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.825312 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:35937: BlockingRecv error: recv got EOF from 127.6.57.65:35937 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:52 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:52.935192 15884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:52.930355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:37899)
0806 06:46:52.930545 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:52.930559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:52.930628 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:52.930760 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:52.930762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:52.930955 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:52.930999 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.931001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.931690 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.931691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.932216 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:52.932218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.932644 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.932645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:52.932710 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.933216 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:52.933221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:52.935076 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
W20260806 06:46:52.935232 15883 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:52.930473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37899 (local address 127.6.57.126:34261)
0806 06:46:52.930572 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:52.930575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:52.930623 (+ 48us) server_negotiation.cc:407] Connection header received
0806 06:46:52.930689 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:52.930690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:52.930703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:52.930769 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:52.931085 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.931586 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.932307 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:52.932524 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:52.932643 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:52.935167 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:37899: BlockingRecv error: recv got EOF from 127.6.57.66:37899 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:53.064414 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:53.059993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:48481)
0806 06:46:53.060128 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:53.060138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:53.060184 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:53.060322 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:53.060323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:53.060484 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:53.060529 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.060531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.061168 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.061169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.061698 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.061700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.062019 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.062019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.062057 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.062565 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:53.062570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:53.064344 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
W20260806 06:46:53.064505 15883 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:53.060088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:48481 (local address 127.6.57.126:34261)
0806 06:46:53.060149 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:53.060151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:53.060169 (+ 18us) server_negotiation.cc:407] Connection header received
0806 06:46:53.060242 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:53.060242 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:53.060254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:53.060306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:53.060587 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.061088 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.061773 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.061952 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.062006 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.064437 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:48481: BlockingRecv error: recv got EOF from 127.6.57.67:48481 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:53.830583 15886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:53.825841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:47249)
0806 06:46:53.825999 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:53.826009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:53.826064 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:53.826192 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:53.826192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:53.826359 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:53.826407 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.826409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.827036 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.827054 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.827579 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.827581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.828151 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.828152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.828217 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.828652 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:53.828656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:53.830456 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
W20260806 06:46:53.830677 15887 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:53.825935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:47249 (local address 127.6.57.126:34261)
0806 06:46:53.826072 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:53.826074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:53.826084 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:53.826108 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:53.826109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:53.826123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:53.826180 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:53.826469 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.826969 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.827701 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.828046 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.828116 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.830602 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:47249: BlockingRecv error: recv got EOF from 127.6.57.65:47249 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:53 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:53.940436 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:53.935756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:46597)
0806 06:46:53.935945 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:53.935956 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:53.936038 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:53.936209 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:53.936210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:53.936377 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:53.936424 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.936425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.937153 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.937155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.937697 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:53.937698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.938071 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.938072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:53.938112 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.938677 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:53.938682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:53.940370 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
W20260806 06:46:53.940562 15887 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:53.935894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:46597 (local address 127.6.57.126:34261)
0806 06:46:53.935961 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:53.935963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:53.936002 (+ 39us) server_negotiation.cc:407] Connection header received
0806 06:46:53.936108 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:53.936109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:53.936123 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:53.936174 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:53.936480 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.937034 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.937790 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:53.937974 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:53.938023 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:53.940459 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:46597: BlockingRecv error: recv got EOF from 127.6.57.66:46597 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:54.069883 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:54.064952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:43897)
0806 06:46:54.065132 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:54.065146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:54.065218 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:54.065364 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:54.065364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:54.065528 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:54.065610 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.065611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.066326 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.066328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.066904 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.066907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.067304 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.067305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.067364 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.067834 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:54.067838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:54.069775 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":134,"threads_started":1}
W20260806 06:46:54.069960 15887 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:54.065000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:43897 (local address 127.6.57.126:34261)
0806 06:46:54.065064 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:54.065066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:54.065193 (+ 127us) server_negotiation.cc:407] Connection header received
0806 06:46:54.065288 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:54.065288 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:54.065300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:54.065339 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:54.065670 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.066239 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.066988 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.067215 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.067301 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.069879 (+ 2578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:43897: BlockingRecv error: recv got EOF from 127.6.57.67:43897 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:54.836045 15890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:54.831132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:44053)
0806 06:46:54.831285 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:54.831297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:54.831370 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:54.831551 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:54.831553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:54.831721 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:54.831768 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.831769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.832394 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.832394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.832917 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.832918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.833294 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.833295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.833332 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.833791 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:54.833795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:54.835943 (+ 2148us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
W20260806 06:46:54.836226 15891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:54.831187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:44053 (local address 127.6.57.126:34261)
0806 06:46:54.831323 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:54.831325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:54.831355 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:46:54.831428 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:54.831428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:54.831439 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:54.831513 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:54.831835 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.832329 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.833029 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.833227 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.833284 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.836115 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:44053: BlockingRecv error: recv got EOF from 127.6.57.65:44053 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:54 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:54.945688 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:54.940915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:42943)
0806 06:46:54.941078 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:54.941089 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:54.941151 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:54.941264 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:54.941265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:54.941446 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:54.941521 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.941522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.942156 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.942157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.942732 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:54.942734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.943276 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.943276 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:54.943318 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.943850 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:54.943854 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:54.945630 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
W20260806 06:46:54.945904 15891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:54.940972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:42943 (local address 127.6.57.126:34261)
0806 06:46:54.941040 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:54.941043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:54.941149 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:46:54.941207 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:54.941207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:54.941219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:54.941258 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:54.941575 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.942078 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.942804 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:54.943195 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:54.943276 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:54.945805 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:42943: BlockingRecv error: recv got EOF from 127.6.57.66:42943 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.67@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:55.075350 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:55.070359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.67:56685)
0806 06:46:55.070529 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:55.070543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:55.070613 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:55.070791 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:55.070792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:55.070972 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:55.071017 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.071019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.071749 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.071750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.072265 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.072266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.072659 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.072660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.072721 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.073173 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:55.073177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:55.075257 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
W20260806 06:46:55.075404 15891 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:55.070505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:56685 (local address 127.6.57.126:34261)
0806 06:46:55.070558 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:55.070560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:55.070593 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:46:55.070701 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:55.070702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:55.070719 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:55.070781 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:55.071096 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.071656 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.072351 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.072565 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.072657 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.075338 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.67:56685: BlockingRecv error: recv got EOF from 127.6.57.67:56685 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.65@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:55.841356 15894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:55.836601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.65:39091)
0806 06:46:55.836756 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:55.836774 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:55.836822 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:55.837008 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:55.837010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:55.837190 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:55.837264 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.837266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.837999 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.838000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.838543 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.838544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.838869 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.838869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.838910 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.839445 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:55.839453 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:55.841282 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
W20260806 06:46:55.841454 15895 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:55.836677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:39091 (local address 127.6.57.126:34261)
0806 06:46:55.836822 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:55.836824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:55.836867 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:46:55.836889 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:55.836890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:55.836900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:55.836976 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:55.837339 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.837912 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.838606 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.838804 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.838857 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.841380 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.65:39091: BlockingRecv error: recv got EOF from 127.6.57.65:39091 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 06:46:55 dist-test-slave-5l3k krb5kdc[14804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.57.66@KRBTEST.COM for oryx/127.6.57.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 06:46:55.951118 15896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:55.946217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34261 (local address 127.6.57.66:37449)
0806 06:46:55.946391 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:55.946406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:55.946503 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:55.946689 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:55.946690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:55.946877 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:55.946924 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.946926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.947648 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.947649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.948181 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:55.948183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.948568 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.948569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:55.948631 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.949114 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:55.949119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:55.951018 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.57.126:34261: Server oryx/127.6.57.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
W20260806 06:46:55.951205 15895 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:55.946317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37449 (local address 127.6.57.126:34261)
0806 06:46:55.946396 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:55.946398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:55.946454 (+ 56us) server_negotiation.cc:407] Connection header received
0806 06:46:55.946593 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:55.946594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:55.946608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:55.946679 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:55.947004 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.947550 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.948270 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:55.948473 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:55.948568 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:55.951136 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.57.66:37449: BlockingRecv error: recv got EOF from 127.6.57.66:37449 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:46:55.993886 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15406
I20260806 06:46:56.000908 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15563
I20260806 06:46:56.005985 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15698
I20260806 06:46:56.010871 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15332
2026-08-06T06:46:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17502 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T06:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:56Z Disabled control of system clock
I20260806 06:46:56.041033 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45027
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:45027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 06:46:56.122663 15900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.122851 15900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.122871 15900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.123986 15900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:56.124020 15900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.124033 15900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.124053 15900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:56.124065 15900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:56.125440 15900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:45027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.125641 15900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.125873 15900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:46:56.130805 15900 server_base.cc:1034] running on GCE node
W20260806 06:46:56.130759 15906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.130820 15908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.130764 15905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.131214 15900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.131521 15900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.132694 15900 hybrid_clock.cc:648] HybridClock initialized: now 1785998816132671 us; error 35 us; skew 500 ppm
I20260806 06:46:56.133873 15900 webserver.cc:460] Webserver started at http://127.6.57.126:41291/ using document root <none> and password file <none>
I20260806 06:46:56.134137 15900 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.134209 15900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.134357 15900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.135423 15900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "5d43018a72f648fcbce9cb47ba9294c2"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "80a4f333b42bbf00aa9c810415016f5f"
server_key_iv: "398c57bc72d53f1da2963b6d3a85dfff"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.135782 15900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "5d43018a72f648fcbce9cb47ba9294c2"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "80a4f333b42bbf00aa9c810415016f5f"
server_key_iv: "398c57bc72d53f1da2963b6d3a85dfff"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.137020 15900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:56.137718 15914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.137907 15900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:56.138010 15900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "5d43018a72f648fcbce9cb47ba9294c2"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "80a4f333b42bbf00aa9c810415016f5f"
server_key_iv: "398c57bc72d53f1da2963b6d3a85dfff"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.138098 15900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.147583 15900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.147840 15900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.148005 15900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.152173 15900 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:45027
I20260806 06:46:56.152204 15968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:45027 every 8 connection(s)
I20260806 06:46:56.152518 15900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:56.153247 15969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:56.155203 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15900
I20260806 06:46:56.155303 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:56.155531 6373 external_mini_cluster.cc:1428] Setting key aa8ed9199e01952a80b6ab2e3f2b4575
I20260806 06:46:56.156091 15969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2: Bootstrap starting.
I20260806 06:46:56.156713 15969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:56.157032 15969 log.cc:824] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:56.157714 15969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2: No bootstrap required, opened a new log
I20260806 06:46:56.159547 15969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } }
I20260806 06:46:56.159693 15969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:56.159744 15969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d43018a72f648fcbce9cb47ba9294c2, State: Initialized, Role: FOLLOWER
I20260806 06:46:56.160193 15969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } }
I20260806 06:46:56.160311 15969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:56.160352 15969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:56.160399 15969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:56.160897 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.156079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.6.57.126:45027)
0806 06:46:56.156273 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.156287 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.156308 (+ 21us) server_negotiation.cc:407] Connection header received
0806 06:46:56.156765 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.156772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.156867 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.157033 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.157254 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.158190 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.159048 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.159438 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.159577 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.160172 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.160177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.160177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.160383 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.160425 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.160440 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.160532 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.160619 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.160697 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260806 06:46:56.161060 15969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } }
I20260806 06:46:56.161180 15969 leader_election.cc:302] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d43018a72f648fcbce9cb47ba9294c2; no voters:
I20260806 06:46:56.161352 15969 leader_election.cc:288] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:56.161397 15975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:56.161538 15975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [term 1 LEADER]: Becoming Leader. State: Replica: 5d43018a72f648fcbce9cb47ba9294c2, State: Running, Role: LEADER
I20260806 06:46:56.161592 15969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:56.161679 15975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } }
I20260806 06:46:56.162120 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d43018a72f648fcbce9cb47ba9294c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } } }
I20260806 06:46:56.162120 15976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d43018a72f648fcbce9cb47ba9294c2. Latest consensus state: current_term: 1 leader_uuid: "5d43018a72f648fcbce9cb47ba9294c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d43018a72f648fcbce9cb47ba9294c2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45027 } } }
I20260806 06:46:56.162225 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:56.162236 15976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:56.162465 15984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:56.162992 15984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:56.164618 15984 catalog_manager.cc:1347] Generated new cluster ID: ea42859d794d4c738b00b65295a522c0
I20260806 06:46:56.164674 15984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:56.172518 15984 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:56.173062 15984 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:56.180341 15984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d43018a72f648fcbce9cb47ba9294c2: Generated new TSK 0
I20260806 06:46:56.180531 15984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:56.189036 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--builtin_ntp_servers=127.6.57.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 06:46:56.271456 15994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.271653 15994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.271674 15994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.272843 15994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.272881 15994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.272904 15994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:56.274228 15994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.274482 15994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.274821 15994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.280153 15999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.280315 15994 server_base.cc:1034] running on GCE node
W20260806 06:46:56.280155 16000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.280441 16002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.280683 15994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.280966 15994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.282135 15994 hybrid_clock.cc:648] HybridClock initialized: now 1785998816282122 us; error 31 us; skew 500 ppm
I20260806 06:46:56.283448 15994 webserver.cc:460] Webserver started at http://127.6.57.65:36331/ using document root <none> and password file <none>
I20260806 06:46:56.283725 15994 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.283795 15994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.283926 15994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.285077 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "306fb7fafa4b4fbeb8745c7733b2d728"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "372385cd853fd48c44f77c4dcc7fa422"
server_key_iv: "67a8456acffcb42e650d03ea7a58afbd"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.285449 15994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "306fb7fafa4b4fbeb8745c7733b2d728"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "372385cd853fd48c44f77c4dcc7fa422"
server_key_iv: "67a8456acffcb42e650d03ea7a58afbd"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.286865 15994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:56.287623 16008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.287806 15994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:56.287909 15994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "306fb7fafa4b4fbeb8745c7733b2d728"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "372385cd853fd48c44f77c4dcc7fa422"
server_key_iv: "67a8456acffcb42e650d03ea7a58afbd"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.287979 15994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.325194 15994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.325528 15994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.325682 15994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.325912 15994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:56.326267 15994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:56.326323 15994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.326356 15994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:56.326382 15994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.332301 15994 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:34469
I20260806 06:46:56.332350 16123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:34469 every 8 connection(s)
I20260806 06:46:56.332718 15994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:56.335237 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 15994
I20260806 06:46:56.335335 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:56.335510 6373 external_mini_cluster.cc:1428] Setting key 1d09afe7af15fea66edd5667e6558e08
I20260806 06:46:56.336472 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--builtin_ntp_servers=127.6.57.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:46:56.338160 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.333437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:51265 (local address 127.6.57.126:45027)
0806 06:46:56.333509 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.333511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.333777 (+ 266us) server_negotiation.cc:407] Connection header received
0806 06:46:56.334276 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.334277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.334291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.334343 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.334967 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.335517 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.336560 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.336810 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.336900 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.337768 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.337770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.337770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.337775 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.337790 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.337794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.337860 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.338022 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.338052 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:46:56.338361 16126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.333432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45027 (local address 127.6.57.65:51265)
0806 06:46:56.333631 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:56.333677 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:56.334174 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:56.334394 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:56.334396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:56.334405 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:56.334894 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.334899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.335614 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.335615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.336457 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.336460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.336912 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.336913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.337055 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.337608 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:56.337608 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:56.337638 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:56.337660 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:56.337900 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:56.337908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:56.337912 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:56.338008 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:46:56.338126 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260806 06:46:56.338891 16124 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45027
I20260806 06:46:56.338999 16124 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:56.339208 16124 heartbeater.cc:511] Master 127.6.57.126:45027 requested a full tablet report, sending...
I20260806 06:46:56.339771 15931 ts_manager.cc:194] Registered new tserver with Master: 306fb7fafa4b4fbeb8745c7733b2d728 (127.6.57.65:34469)
I20260806 06:46:56.340767 15931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.65:51265
W20260806 06:46:56.419050 16127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.419236 16127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.419256 16127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.420302 16127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.420331 16127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.420351 16127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:56.421487 16127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.421655 16127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.421867 16127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.426690 16133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.426697 16132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.426735 16127 server_base.cc:1034] running on GCE node
W20260806 06:46:56.426908 16135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.427096 16127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.427371 16127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.428584 16127 hybrid_clock.cc:648] HybridClock initialized: now 1785998816428546 us; error 39 us; skew 500 ppm
I20260806 06:46:56.429827 16127 webserver.cc:460] Webserver started at http://127.6.57.66:40297/ using document root <none> and password file <none>
I20260806 06:46:56.430058 16127 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.430146 16127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.430306 16127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.431361 16127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "443f7fcbeb244cf7886f765e37a23f64"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49c105a9930d863e1a7ae363a772d413"
server_key_iv: "6ffc5666513a13c96dd0f4622ca14916"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.431725 16127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "443f7fcbeb244cf7886f765e37a23f64"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49c105a9930d863e1a7ae363a772d413"
server_key_iv: "6ffc5666513a13c96dd0f4622ca14916"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.433053 16127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:56.433749 16141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.433910 16127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:56.434013 16127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "443f7fcbeb244cf7886f765e37a23f64"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49c105a9930d863e1a7ae363a772d413"
server_key_iv: "6ffc5666513a13c96dd0f4622ca14916"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.434087 16127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.452821 16127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.453102 16127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.453258 16127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.453467 16127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:56.453830 16127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:56.453873 16127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.453893 16127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:56.453936 16127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.459762 16127 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:35001
I20260806 06:46:56.459807 16257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:35001 every 8 connection(s)
I20260806 06:46:56.460126 16127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:56.462651 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16127
I20260806 06:46:56.462814 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:56.463014 6373 external_mini_cluster.cc:1428] Setting key 63eb2f83b927ac143050c9498d58fe39
I20260806 06:46:56.464061 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--builtin_ntp_servers=127.6.57.84:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 06:46:56.465855 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.460834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:34335 (local address 127.6.57.126:45027)
0806 06:46:56.460896 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.460898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.461137 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:46:56.461688 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.461689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.461703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.461761 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.462451 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.463198 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.464361 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.464586 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.464670 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.465572 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.465574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.465575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.465581 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.465600 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.465605 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.465634 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.465774 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.465798 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:46:56.466118 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.460833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45027 (local address 127.6.57.66:34335)
0806 06:46:56.461007 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:56.461039 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:56.461576 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:56.461841 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:56.461845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:56.461861 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:56.462314 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.462318 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.463292 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.463293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.464223 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.464226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.464690 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.464691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.464823 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.465378 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:56.465379 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:56.465422 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:56.465455 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:56.465678 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:56.465683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:56.465686 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:56.465754 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:46:56.465864 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260806 06:46:56.466643 16258 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45027
I20260806 06:46:56.466735 16258 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:56.466919 16258 heartbeater.cc:511] Master 127.6.57.126:45027 requested a full tablet report, sending...
I20260806 06:46:56.467341 15931 ts_manager.cc:194] Registered new tserver with Master: 443f7fcbeb244cf7886f765e37a23f64 (127.6.57.66:35001)
I20260806 06:46:56.467846 15931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.66:34335
W20260806 06:46:56.546867 16261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.547050 16261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.547070 16261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.548153 16261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.548187 16261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.548210 16261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:56.549468 16261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:45027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.549664 16261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.549960 16261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.554744 16267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.554894 16261 server_base.cc:1034] running on GCE node
W20260806 06:46:56.554752 16269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.554967 16266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.555243 16261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.555471 16261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.556659 16261 hybrid_clock.cc:648] HybridClock initialized: now 1785998816556640 us; error 35 us; skew 500 ppm
I20260806 06:46:56.557909 16261 webserver.cc:460] Webserver started at http://127.6.57.67:33333/ using document root <none> and password file <none>
I20260806 06:46:56.558197 16261 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.558246 16261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.558394 16261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.559433 16261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "e77f2e07ab9c455ca41995f0b867ccba"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49dda1779f7b336224dcfaa5ba0fa582"
server_key_iv: "be38ebc3f0e49783a494b954ab244392"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.559780 16261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "e77f2e07ab9c455ca41995f0b867ccba"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49dda1779f7b336224dcfaa5ba0fa582"
server_key_iv: "be38ebc3f0e49783a494b954ab244392"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.561223 16261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:56.562047 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.562292 16261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:56.562364 16261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "e77f2e07ab9c455ca41995f0b867ccba"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "49dda1779f7b336224dcfaa5ba0fa582"
server_key_iv: "be38ebc3f0e49783a494b954ab244392"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.562451 16261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.572541 16261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.572798 16261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.572937 16261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.573138 16261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:56.573441 16261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:56.573472 16261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.573520 16261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:56.573547 16261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.579387 16261 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:40013
I20260806 06:46:56.579435 16393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:40013 every 8 connection(s)
I20260806 06:46:56.579722 16261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:56.580282 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16261
I20260806 06:46:56.580375 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:56.580605 6373 external_mini_cluster.cc:1428] Setting key 63f78b5db55119480ef6d08f90258fa8
I20260806 06:46:56.584930 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.580795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:51085 (local address 127.6.57.126:45027)
0806 06:46:56.580868 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.580870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.580882 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:56.581471 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.581472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.581484 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.581542 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.582199 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.582787 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.583787 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.583971 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.584019 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.584718 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.584719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.584719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.584723 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.584736 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.584740 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.584781 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.584857 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.584875 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:46:56.585139 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.580409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:45027 (local address 127.6.57.67:51085)
0806 06:46:56.580616 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:56.580668 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:56.581359 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:56.581609 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:56.581614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:56.581624 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:56.582121 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.582125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.582863 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.582864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.583716 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:56.583718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:56.584063 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:56.584064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:56.584163 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:56.584599 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:56.584599 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:56.584623 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:56.584643 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:56.584792 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:56.584795 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:56.584797 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:56.584846 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:46:56.584917 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260806 06:46:56.585534 16394 heartbeater.cc:347] Connected to a master server at 127.6.57.126:45027
I20260806 06:46:56.585625 16394 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:56.585794 16394 heartbeater.cc:511] Master 127.6.57.126:45027 requested a full tablet report, sending...
I20260806 06:46:56.586153 15931 ts_manager.cc:194] Registered new tserver with Master: e77f2e07ab9c455ca41995f0b867ccba (127.6.57.67:40013)
I20260806 06:46:56.586540 15931 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.67:51085
I20260806 06:46:56.592409 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:46:56.594205 15973 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:56.593692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.6.57.126:45027)
0806 06:46:56.593743 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.593745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.593862 (+ 117us) server_negotiation.cc:407] Connection header received
0806 06:46:56.593936 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.593937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.593946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.594012 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.594137 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52152: BlockingRecv error: recv got EOF from 127.0.0.1:52152 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:46:56.594300 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15994
I20260806 06:46:56.600034 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16127
I20260806 06:46:56.605818 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16261
I20260806 06:46:56.611092 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 15900
2026-08-06T06:46:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (599 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T06:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:56Z Disabled control of system clock
I20260806 06:46:56.639829 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41101
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:46:56.723007 16405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.723194 16405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.723214 16405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.724277 16405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:56.724309 16405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.724321 16405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.724334 16405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:56.724346 16405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:56.725613 16405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.725787 16405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.726023 16405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.730815 16414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.730917 16410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.730912 16411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.731158 16405 server_base.cc:1034] running on GCE node
I20260806 06:46:56.731400 16405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.731642 16405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.732790 16405 hybrid_clock.cc:648] HybridClock initialized: now 1785998816732778 us; error 37 us; skew 500 ppm
I20260806 06:46:56.734099 16405 webserver.cc:460] Webserver started at http://127.6.57.126:36263/ using document root <none> and password file <none>
I20260806 06:46:56.734375 16405 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.734422 16405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.734627 16405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.735700 16405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "d08d8a799077ed82cbe3c71fdfcb4687"
server_key_iv: "152da4b05b592d67a030335050d21a14"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.736140 16405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "d08d8a799077ed82cbe3c71fdfcb4687"
server_key_iv: "152da4b05b592d67a030335050d21a14"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.737439 16405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:56.738168 16420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.738359 16405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:56.738467 16405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "d08d8a799077ed82cbe3c71fdfcb4687"
server_key_iv: "152da4b05b592d67a030335050d21a14"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.738592 16405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.741947 16405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.742223 16405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.742367 16405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.746158 16405 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41101
I20260806 06:46:56.746201 16473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41101 every 8 connection(s)
I20260806 06:46:56.746647 16405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:56.747316 16474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:56.749884 16474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce: Bootstrap starting.
I20260806 06:46:56.750398 16474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:56.750725 16474 log.cc:824] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:56.751308 16474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce: No bootstrap required, opened a new log
I20260806 06:46:56.752805 16474 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } }
I20260806 06:46:56.752941 16474 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:56.752964 16474 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5310df3aa34e2bb9f1c5cf70d359ce, State: Initialized, Role: FOLLOWER
I20260806 06:46:56.753111 16474 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } }
I20260806 06:46:56.753189 16474 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:56.753213 16474 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:56.753240 16474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:56.753722 16474 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } }
I20260806 06:46:56.753824 16474 leader_election.cc:302] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc5310df3aa34e2bb9f1c5cf70d359ce; no voters:
I20260806 06:46:56.753999 16474 leader_election.cc:288] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:56.754029 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16405
I20260806 06:46:56.754169 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:56.754194 16477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:56.754264 16474 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:56.754427 6373 external_mini_cluster.cc:1428] Setting key faa7a053ba5dc7a8e1c9ed35f5e16cad
I20260806 06:46:56.754441 16477 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [term 1 LEADER]: Becoming Leader. State: Replica: fc5310df3aa34e2bb9f1c5cf70d359ce, State: Running, Role: LEADER
I20260806 06:46:56.754565 16477 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } }
I20260806 06:46:56.755155 16479 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc5310df3aa34e2bb9f1c5cf70d359ce. Latest consensus state: current_term: 1 leader_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } } }
I20260806 06:46:56.755196 16478 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5310df3aa34e2bb9f1c5cf70d359ce" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41101 } } }
I20260806 06:46:56.755262 16479 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:56.755268 16478 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:56.755721 16489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:56.756245 16489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:56.757933 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.754967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.6.57.126:41101)
0806 06:46:56.755524 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.755532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.755912 (+ 380us) server_negotiation.cc:407] Connection header received
0806 06:46:56.756081 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.756088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.756136 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.756284 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.757409 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.757412 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.757413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.757417 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.757468 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.757480 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.757580 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.757677 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.757752 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":83,"threads_started":1}
I20260806 06:46:56.758245 16489 catalog_manager.cc:1347] Generated new cluster ID: cc4071fe92534200b8f24df9750a0dc2
I20260806 06:46:56.758296 16489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:46:56.771256 16489 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:56.771427 16489 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:56.776098 16489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc5310df3aa34e2bb9f1c5cf70d359ce: Generated new TSK 0
I20260806 06:46:56.776310 16489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:46:56.778374 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--builtin_ntp_servers=127.6.57.84:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:46:56.868538 16499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.868772 16499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.868810 16499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.869956 16499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.870016 16499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.870057 16499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:56.871459 16499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.871716 16499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.871989 16499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.878183 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.878170 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.878487 16499 server_base.cc:1034] running on GCE node
W20260806 06:46:56.878192 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.878825 16499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.879091 16499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.880247 16499 hybrid_clock.cc:648] HybridClock initialized: now 1785998816880230 us; error 30 us; skew 500 ppm
I20260806 06:46:56.886732 16499 webserver.cc:460] Webserver started at http://127.6.57.65:43773/ using document root <none> and password file <none>
I20260806 06:46:56.886931 16499 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.886999 16499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.887107 16499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.888208 16499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "95b29f6cb32246eeb75e5f6770d21e6e"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "354d8b9e34f2d11aaaa887a41b96d3d0"
server_key_iv: "a91a860311319bc939f9640c9a4415b0"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.888526 16499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "95b29f6cb32246eeb75e5f6770d21e6e"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "354d8b9e34f2d11aaaa887a41b96d3d0"
server_key_iv: "a91a860311319bc939f9640c9a4415b0"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.889822 16499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:56.890759 16514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.890950 16499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:56.891088 16499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "95b29f6cb32246eeb75e5f6770d21e6e"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "354d8b9e34f2d11aaaa887a41b96d3d0"
server_key_iv: "a91a860311319bc939f9640c9a4415b0"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.891168 16499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:56.894313 16499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:56.894631 16499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:56.894775 16499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:56.894995 16499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:56.895313 16499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:56.895385 16499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.895426 16499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:56.895452 16499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:56.901222 16499 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39243
I20260806 06:46:56.901258 16629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39243 every 8 connection(s)
I20260806 06:46:56.901569 16499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:46:56.904479 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.902327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:44731 (local address 127.6.57.126:41101)
0806 06:46:56.902395 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:56.902397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:56.902646 (+ 249us) server_negotiation.cc:407] Connection header received
0806 06:46:56.903099 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:56.903100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:56.903108 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:56.903163 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:56.904315 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:56.904316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:56.904316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:56.904319 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:56.904329 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:56.904332 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:56.904372 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:56.904416 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 06:46:56.904428 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:46:56.904675 16632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:56.902312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41101 (local address 127.6.57.65:44731)
0806 06:46:56.902536 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:56.902580 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:56.902997 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:56.903237 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:56.903241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:56.903249 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:56.904189 (+ 940us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:56.904190 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:56.904217 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:56.904244 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:56.904371 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:56.904373 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:56.904375 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:46:56.904424 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:46:56.904468 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260806 06:46:56.905210 16630 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41101
I20260806 06:46:56.905298 16630 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:56.905401 16630 heartbeater.cc:511] Master 127.6.57.126:41101 requested a full tablet report, sending...
I20260806 06:46:56.905674 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16499
I20260806 06:46:56.905782 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:56.905993 6373 external_mini_cluster.cc:1428] Setting key 1f67a1b41ed8fb308082ad8e31bcf9fa
I20260806 06:46:56.905982 16437 ts_manager.cc:194] Registered new tserver with Master: 95b29f6cb32246eeb75e5f6770d21e6e (127.6.57.65:39243)
I20260806 06:46:56.906895 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--builtin_ntp_servers=127.6.57.84:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:46:56.986971 16633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:56.987160 16633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:56.987181 16633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:56.988312 16633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:56.988349 16633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:56.988374 16633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:56.989627 16633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:56.989794 16633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:56.990034 16633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:56.994902 16639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.994889 16638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:56.994901 16641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:56.994983 16633 server_base.cc:1034] running on GCE node
I20260806 06:46:56.995394 16633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:56.995630 16633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:56.996793 16633 hybrid_clock.cc:648] HybridClock initialized: now 1785998816996780 us; error 35 us; skew 500 ppm
I20260806 06:46:56.997984 16633 webserver.cc:460] Webserver started at http://127.6.57.66:44181/ using document root <none> and password file <none>
I20260806 06:46:56.998221 16633 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:56.998266 16633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:56.998436 16633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:56.999509 16633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "c74269e6022a4c7d8d09ead633602acb"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "4cfd1b522c44dea7e6d3fa86840ce728"
server_key_iv: "405b0923657fb61b783bd719b02e4d50"
server_key_version: "encryptionkey@0"
I20260806 06:46:56.999873 16633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "c74269e6022a4c7d8d09ead633602acb"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "4cfd1b522c44dea7e6d3fa86840ce728"
server_key_iv: "405b0923657fb61b783bd719b02e4d50"
server_key_version: "encryptionkey@0"
I20260806 06:46:57.001145 16633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:57.001904 16647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.002058 16633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:57.002147 16633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "c74269e6022a4c7d8d09ead633602acb"
format_stamp: "Formatted at 2026-08-06 06:46:56 on dist-test-slave-5l3k"
server_key: "4cfd1b522c44dea7e6d3fa86840ce728"
server_key_iv: "405b0923657fb61b783bd719b02e4d50"
server_key_version: "encryptionkey@0"
I20260806 06:46:57.002203 16633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:57.005329 16633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:57.005525 16633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:57.005614 16633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:57.005759 16633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:57.005987 16633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:57.006017 16633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.006036 16633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:57.006048 16633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.011621 16633 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:41621
I20260806 06:46:57.011690 16765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:41621 every 8 connection(s)
I20260806 06:46:57.011957 16633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:46:57.014891 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:57.012631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:49467 (local address 127.6.57.126:41101)
0806 06:46:57.012676 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:57.012678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:57.012929 (+ 251us) server_negotiation.cc:407] Connection header received
0806 06:46:57.013440 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:57.013442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:57.013450 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:57.013502 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:57.014636 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:57.014637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:57.014638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:57.014643 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:57.014660 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:57.014664 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:57.014722 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:57.014807 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 06:46:57.014823 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:46:57.015084 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:57.012635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41101 (local address 127.6.57.66:49467)
0806 06:46:57.012818 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:57.012848 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:57.013306 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:57.013564 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:57.013568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:57.013578 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:57.014443 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:57.014444 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:57.014469 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:57.014541 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:57.014730 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:57.014735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:57.014738 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:57.014808 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:46:57.014869 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260806 06:46:57.015574 16766 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41101
I20260806 06:46:57.015645 16766 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:57.015799 16766 heartbeater.cc:511] Master 127.6.57.126:41101 requested a full tablet report, sending...
I20260806 06:46:57.016155 16437 ts_manager.cc:194] Registered new tserver with Master: c74269e6022a4c7d8d09ead633602acb (127.6.57.66:41621)
I20260806 06:46:57.022004 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16633
I20260806 06:46:57.022113 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:57.022293 6373 external_mini_cluster.cc:1428] Setting key 66d73178066ef48dccf9d0acae26cd02
I20260806 06:46:57.023123 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--builtin_ntp_servers=127.6.57.84:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 06:46:57.102672 16769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:57.102861 16769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:57.102882 16769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:57.104041 16769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:57.104080 16769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:57.104105 16769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:57.105556 16769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:57.105808 16769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:57.106093 16769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:57.110780 16777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:57.110782 16775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:57.110807 16774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:57.111076 16769 server_base.cc:1034] running on GCE node
I20260806 06:46:57.111310 16769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:57.111542 16769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:57.112699 16769 hybrid_clock.cc:648] HybridClock initialized: now 1785998817112704 us; error 55 us; skew 500 ppm
I20260806 06:46:57.113952 16769 webserver.cc:460] Webserver started at http://127.6.57.67:38655/ using document root <none> and password file <none>
I20260806 06:46:57.114195 16769 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:57.114317 16769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:57.114533 16769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:57.115558 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "72075ec141544f2ab224e34f854a10ac"
format_stamp: "Formatted at 2026-08-06 06:46:57 on dist-test-slave-5l3k"
server_key: "8ad0ad2ad3b90b0d53b72cf3d4ca5493"
server_key_iv: "3b601b1a1f9adaa1a5509c71e3140db9"
server_key_version: "encryptionkey@0"
I20260806 06:46:57.115913 16769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "72075ec141544f2ab224e34f854a10ac"
format_stamp: "Formatted at 2026-08-06 06:46:57 on dist-test-slave-5l3k"
server_key: "8ad0ad2ad3b90b0d53b72cf3d4ca5493"
server_key_iv: "3b601b1a1f9adaa1a5509c71e3140db9"
server_key_version: "encryptionkey@0"
I20260806 06:46:57.117215 16769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:46:57.117964 16783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.118131 16769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:46:57.118230 16769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "72075ec141544f2ab224e34f854a10ac"
format_stamp: "Formatted at 2026-08-06 06:46:57 on dist-test-slave-5l3k"
server_key: "8ad0ad2ad3b90b0d53b72cf3d4ca5493"
server_key_iv: "3b601b1a1f9adaa1a5509c71e3140db9"
server_key_version: "encryptionkey@0"
I20260806 06:46:57.118310 16769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:57.121411 16769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:57.121660 16769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:57.121793 16769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:57.122013 16769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:57.122305 16769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:57.122359 16769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.122394 16769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:57.122419 16769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:57.128057 16769 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:39343
I20260806 06:46:57.128140 16904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:39343 every 8 connection(s)
I20260806 06:46:57.128432 16769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:46:57.131182 16482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:57.129166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:53225 (local address 127.6.57.126:41101)
0806 06:46:57.129233 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:57.129235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:57.129401 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:46:57.129820 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:57.129821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:57.129826 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:57.129881 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:57.130970 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:57.130972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:57.130972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:46:57.130976 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:57.130992 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:46:57.130995 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:57.131042 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:57.131116 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 06:46:57.131131 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:46:57.131361 16907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:57.129124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41101 (local address 127.6.57.67:53225)
0806 06:46:57.129293 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:57.129332 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:57.129740 (+ 408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:57.129927 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:57.129930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:57.129937 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:57.130840 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:46:57.130840 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:57.130867 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:46:57.130892 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:57.131047 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:57.131049 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:57.131052 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:46:57.131101 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:46:57.131146 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260806 06:46:57.131846 16905 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41101
I20260806 06:46:57.131930 16905 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:57.132037 16905 heartbeater.cc:511] Master 127.6.57.126:41101 requested a full tablet report, sending...
I20260806 06:46:57.132380 16437 ts_manager.cc:194] Registered new tserver with Master: 72075ec141544f2ab224e34f854a10ac (127.6.57.67:39343)
I20260806 06:46:57.138227 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16769
I20260806 06:46:57.138293 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:57.138441 6373 external_mini_cluster.cc:1428] Setting key a0fa8700f9932127799d06d9fee07eb9
I20260806 06:46:57.139293 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 06:46:57.141156 16482 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:46:57.140619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.6.57.126:41101)
0806 06:46:57.140693 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:57.140695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:57.140815 (+ 120us) server_negotiation.cc:407] Connection header received
0806 06:46:57.140904 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:57.140905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:57.140910 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:57.140966 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:57.141095 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40682: BlockingRecv error: recv got EOF from 127.0.0.1:40682 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 06:46:57.141311 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16499
I20260806 06:46:57.147851 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16633
I20260806 06:46:57.152959 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16769
I20260806 06:46:57.157569 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16405
2026-08-06T06:46:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (546 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:57 dist-test-slave-5l3k krb5kdc[16915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:57 dist-test-slave-5l3k krb5kdc[16915](info): set up 2 sockets
Aug 06 06:46:57 dist-test-slave-5l3k krb5kdc[16915](info): commencing operation
krb5kdc: starting...
W20260806 06:46:58.217722 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:46:58.339226 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43791
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43791
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:58.421340 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:58.421533 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:58.421554 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:58.422730 16928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:46:58.422770 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:58.422784 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:58.422796 16928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:46:58.422868 16928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:46:58.424338 16928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:58.424584 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:58.424806 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:58.429586 16933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.429594 16934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.429623 16936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:58.429739 16928 server_base.cc:1034] running on GCE node
I20260806 06:46:58.430011 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:58.430302 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:58.431438 16928 hybrid_clock.cc:648] HybridClock initialized: now 1785998818431413 us; error 38 us; skew 500 ppm
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:58.433943 16928 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:46:58.434355 16928 webserver.cc:460] Webserver started at http://127.6.57.126:32911/ using document root <none> and password file <none>
I20260806 06:46:58.434660 16928 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:58.434731 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:58.434875 16928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:58.435963 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "bf03a65f78b44d318f10978587c20ce1"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "d390b715dd55c93bc087d2665b20f9ec"
server_key_iv: "9db68703c2ad3ab6333e087403a901e4"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.436317 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "bf03a65f78b44d318f10978587c20ce1"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "d390b715dd55c93bc087d2665b20f9ec"
server_key_iv: "9db68703c2ad3ab6333e087403a901e4"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.437638 16928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:46:58.438367 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.438611 16928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:58.438707 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "bf03a65f78b44d318f10978587c20ce1"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "d390b715dd55c93bc087d2665b20f9ec"
server_key_iv: "9db68703c2ad3ab6333e087403a901e4"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.438798 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:58.451218 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:58.452934 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:58.453094 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:58.457106 16928 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:43791
I20260806 06:46:58.457175 16997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:43791 every 8 connection(s)
I20260806 06:46:58.457576 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:46:58.458662 16998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:58.460808 16998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: Bootstrap starting.
I20260806 06:46:58.461329 16998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:58.461582 16998 log.cc:824] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:58.462262 16998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: No bootstrap required, opened a new log
I20260806 06:46:58.463721 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 16928
I20260806 06:46:58.463832 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:46:58.464035 6373 external_mini_cluster.cc:1428] Setting key f9ba9d3ff77fe311eaadf84c710ad3c6
I20260806 06:46:58.463960 16998 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } }
I20260806 06:46:58.464102 16998 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:58.464165 16998 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf03a65f78b44d318f10978587c20ce1, State: Initialized, Role: FOLLOWER
I20260806 06:46:58.464675 16998 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } }
I20260806 06:46:58.464782 16998 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:46:58.464838 16998 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:46:58.464886 16998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:58.465600 16998 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } }
I20260806 06:46:58.465731 16998 leader_election.cc:302] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf03a65f78b44d318f10978587c20ce1; no voters:
I20260806 06:46:58.465910 16998 leader_election.cc:288] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:46:58.465950 17003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:58.466095 17003 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [term 1 LEADER]: Becoming Leader. State: Replica: bf03a65f78b44d318f10978587c20ce1, State: Running, Role: LEADER
I20260806 06:46:58.466161 16998 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:46:58.466185 17003 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } }
I20260806 06:46:58.466553 17003 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf03a65f78b44d318f10978587c20ce1. Latest consensus state: current_term: 1 leader_uuid: "bf03a65f78b44d318f10978587c20ce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } } }
I20260806 06:46:58.466634 17003 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:58.466557 17004 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf03a65f78b44d318f10978587c20ce1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf03a65f78b44d318f10978587c20ce1" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43791 } } }
I20260806 06:46:58.466776 17004 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1 [sys.catalog]: This master's current role is: LEADER
I20260806 06:46:58.467476 17011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:46:58.468261 17011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:46:58.470651 17011 catalog_manager.cc:1347] Generated new cluster ID: a69d59d7da2c4fa2a3bf7fe3be068163
I20260806 06:46:58.470706 17011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:58.474684 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:58.464569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.6.57.126:43791)
0806 06:46:58.464766 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:58.464792 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:58.464826 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:46:58.464940 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:58.464946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:58.465002 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:58.465142 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:58.465920 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.467262 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.468213 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.468702 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.468829 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:58.471413 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:58.471416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:58.471417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:58.471448 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:58.473164 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.473531 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.473534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.473534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.473641 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.473854 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.473855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.473856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.474080 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:58.474263 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:58.474328 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 06:46:58.474469 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260806 06:46:58.483059 17011 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:46:58.483572 17011 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:46:58.486532 17011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: Generated new TSK 0
I20260806 06:46:58.486729 17011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:46:58.526589 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:58.631940 17026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:58.632148 17026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:58.632171 17026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:58.633320 17026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:58.633361 17026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:58.633387 17026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:46:58.634855 17026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:58.635063 17026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:58.635238 17026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:58.640334 17031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.640332 17034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.640339 17032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:58.640437 17026 server_base.cc:1034] running on GCE node
I20260806 06:46:58.640796 17026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:58.641057 17026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:58.642231 17026 hybrid_clock.cc:648] HybridClock initialized: now 1785998818642214 us; error 33 us; skew 500 ppm
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:58.644790 17026 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:46:58.645253 17026 webserver.cc:460] Webserver started at http://127.6.57.65:33109/ using document root <none> and password file <none>
I20260806 06:46:58.645517 17026 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:58.645592 17026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:58.645742 17026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:58.646921 17026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "4171d65aabda4e3c96c1a31204592b8a"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "5b023379f2f5b51b72fe3a5c945a12e2"
server_key_iv: "7105e46452a4c5285f11c342dd40a186"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.647296 17026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "4171d65aabda4e3c96c1a31204592b8a"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "5b023379f2f5b51b72fe3a5c945a12e2"
server_key_iv: "7105e46452a4c5285f11c342dd40a186"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.648622 17026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:46:58.649326 17041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.649493 17026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:46:58.649609 17026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "4171d65aabda4e3c96c1a31204592b8a"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "5b023379f2f5b51b72fe3a5c945a12e2"
server_key_iv: "7105e46452a4c5285f11c342dd40a186"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.649685 17026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:58.667927 17026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:58.669804 17026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:58.669975 17026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:58.670219 17026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:58.670594 17026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:58.670655 17026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.670688 17026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:58.670715 17026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.676458 17026 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:37351
I20260806 06:46:58.676554 17159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:37351 every 8 connection(s)
I20260806 06:46:58.676867 17026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:58.683288 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 17026
I20260806 06:46:58.683427 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:46:58.683701 6373 external_mini_cluster.cc:1428] Setting key 71281953d8df9f3158d41076be7038c8
I20260806 06:46:58.686390 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:58.677615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:46537 (local address 127.6.57.126:43791)
0806 06:46:58.677680 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:58.677683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:58.677950 (+ 267us) server_negotiation.cc:407] Connection header received
0806 06:46:58.678442 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:58.678443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:58.678458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:58.678559 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:58.679482 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.679983 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.680942 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.681134 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.681184 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:58.683384 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:58.683386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:58.683386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:58.683395 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:58.684947 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.685455 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.685456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.685456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.685507 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.685780 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.685781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.685781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.685878 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:58.685927 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:58.686260 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 06:46:58.686326 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:46:58.686720 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:58.677597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43791 (local address 127.6.57.65:46537)
0806 06:46:58.677797 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:58.677834 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:58.678353 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:58.678632 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:58.678636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:58.679209 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:58.679397 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:58.679402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.680092 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.680093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:58.680865 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:58.680867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.681208 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.681208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:58.681308 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:58.681733 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:58.681741 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:58.683255 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:58.685031 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:58.685047 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:58.685050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:58.685377 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:58.685572 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:58.685573 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:58.685574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:58.685716 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:58.685939 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:58.685946 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:58.686119 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 06:46:58.686235 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:46:58.686423 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260806 06:46:58.687282 17160 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43791
I20260806 06:46:58.687392 17160 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:58.687594 17160 heartbeater.cc:511] Master 127.6.57.126:43791 requested a full tablet report, sending...
I20260806 06:46:58.688194 16960 ts_manager.cc:194] Registered new tserver with Master: 4171d65aabda4e3c96c1a31204592b8a (127.6.57.65:37351)
I20260806 06:46:58.688797 16960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:46537
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:46:58.710819 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:58.795647 17166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:58.795835 17166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:58.795857 17166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:58.797046 17166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:58.797088 17166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:58.797114 17166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:46:58.798617 17166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:58.798888 17166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:58.799127 17166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:58.804450 17171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.804456 17174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.804528 17172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:58.804669 17166 server_base.cc:1034] running on GCE node
I20260806 06:46:58.804908 17166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:58.805159 17166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:58.806303 17166 hybrid_clock.cc:648] HybridClock initialized: now 1785998818806281 us; error 40 us; skew 500 ppm
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:58.808769 17166 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:46:58.809297 17166 webserver.cc:460] Webserver started at http://127.6.57.66:46109/ using document root <none> and password file <none>
I20260806 06:46:58.809532 17166 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:58.809576 17166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:58.809737 17166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:58.810833 17166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "94475ad2714343ae950db63bbfb1c5d0"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "a77bd8311b871d99e729210ceefabaf3"
server_key_iv: "eb8eba1451a7e0f4a3daf8e4edca8b15"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.811183 17166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "94475ad2714343ae950db63bbfb1c5d0"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "a77bd8311b871d99e729210ceefabaf3"
server_key_iv: "eb8eba1451a7e0f4a3daf8e4edca8b15"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.812507 17166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:58.813234 17181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.813468 17166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:46:58.813571 17166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "94475ad2714343ae950db63bbfb1c5d0"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "a77bd8311b871d99e729210ceefabaf3"
server_key_iv: "eb8eba1451a7e0f4a3daf8e4edca8b15"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.813642 17166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:58.830094 17166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:58.831933 17166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:58.832090 17166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:58.832322 17166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:58.832650 17166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:58.832693 17166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.832715 17166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:58.832769 17166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.838917 17166 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:34911
I20260806 06:46:58.838991 17299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:34911 every 8 connection(s)
I20260806 06:46:58.839288 17166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:58.846331 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 17166
I20260806 06:46:58.846426 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:46:58.846710 6373 external_mini_cluster.cc:1428] Setting key 8d51f21b31ad37b3cd030b26c4d090d9
I20260806 06:46:58.849359 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:58.840015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:47697 (local address 127.6.57.126:43791)
0806 06:46:58.840059 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:58.840061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:58.840256 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:46:58.840853 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:58.840855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:58.840869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:58.840925 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:58.841907 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.842418 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.843431 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.843640 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.843729 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:58.845867 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:58.845868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:58.845868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:58.845875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:58.847464 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.848000 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.848001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.848001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.848103 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:58.848455 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:58.848456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:58.848456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:58.848629 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:58.848712 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:58.849106 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 06:46:58.849217 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:46:58.849435 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:58.839979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43791 (local address 127.6.57.66:47697)
0806 06:46:58.840139 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:58.840174 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:58.840702 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:58.840993 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:58.840997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:58.841617 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:58.841801 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:58.841806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.842538 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.842539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:58.843332 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:58.843335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:58.843739 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:58.843740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:58.843865 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:58.844384 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:58.844392 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:58.845781 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:58.847548 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:58.847555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:58.847557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:58.847895 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:58.848214 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:58.848217 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:58.848217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:58.848387 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:58.848717 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:58.848723 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:58.848939 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 06:46:58.849044 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:46:58.849214 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260806 06:46:58.850128 17300 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43791
I20260806 06:46:58.850282 17300 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:58.850556 17300 heartbeater.cc:511] Master 127.6.57.126:43791 requested a full tablet report, sending...
I20260806 06:46:58.851141 16960 ts_manager.cc:194] Registered new tserver with Master: 94475ad2714343ae950db63bbfb1c5d0 (127.6.57.66:34911)
I20260806 06:46:58.851719 16960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:47697
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:46:58.877214 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:46:58.959631 17305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:46:58.959816 17305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:46:58.959839 17305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:46:58.960857 17305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:46:58.960889 17305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:46:58.960914 17305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:46:58.962375 17305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:46:58.962611 17305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:46:58.962786 17305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:46:58.967486 17312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.967461 17314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:46:58.967463 17311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:46:58.968017 17305 server_base.cc:1034] running on GCE node
I20260806 06:46:58.968258 17305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:46:58.968487 17305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:46:58.969647 17305 hybrid_clock.cc:648] HybridClock initialized: now 1785998818969642 us; error 43 us; skew 500 ppm
Aug 06 06:46:58 dist-test-slave-5l3k krb5kdc[16915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:46:58.972121 17305 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:46:58.972468 17305 webserver.cc:460] Webserver started at http://127.6.57.67:35465/ using document root <none> and password file <none>
I20260806 06:46:58.972640 17305 fs_manager.cc:359] Metadata directory not provided
I20260806 06:46:58.972678 17305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:46:58.972760 17305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:46:58.973706 17305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "fa04231a0d354a8cbed3f7780e21e53e"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "721f772dc12e100c0788dde49181612c"
server_key_iv: "90ae088ccbc941510e0879f14e45dea1"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.974006 17305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "fa04231a0d354a8cbed3f7780e21e53e"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "721f772dc12e100c0788dde49181612c"
server_key_iv: "90ae088ccbc941510e0879f14e45dea1"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.975593 17305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:46:58.976359 17321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.976593 17305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:46:58.976677 17305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "fa04231a0d354a8cbed3f7780e21e53e"
format_stamp: "Formatted at 2026-08-06 06:46:58 on dist-test-slave-5l3k"
server_key: "721f772dc12e100c0788dde49181612c"
server_key_iv: "90ae088ccbc941510e0879f14e45dea1"
server_key_version: "encryptionkey@0"
I20260806 06:46:58.976728 17305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:46:58.994159 17305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:46:58.996318 17305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:46:58.996443 17305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:46:58.996611 17305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:46:58.996924 17305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:46:58.996991 17305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:58.997023 17305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:46:58.997054 17305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:46:59.002723 17305 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:41915
I20260806 06:46:59.002797 17438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:41915 every 8 connection(s)
I20260806 06:46:59.003108 17305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:46:59 dist-test-slave-5l3k krb5kdc[16915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998818, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:46:59.012275 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.003837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34947 (local address 127.6.57.126:43791)
0806 06:46:59.003899 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.003916 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.004109 (+ 193us) server_negotiation.cc:407] Connection header received
0806 06:46:59.004639 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.004641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.004653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.004703 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:59.005642 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.006156 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.007170 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.007373 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.007456 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.009486 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:59.009488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:59.009488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:59.009497 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:59.010881 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:59.011292 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:59.011293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:59.011294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:59.011352 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:59.011594 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:59.011595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:59.011595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:59.011711 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:59.011785 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.012058 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.012183 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:46:59.012447 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.003812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:43791 (local address 127.6.57.67:34947)
0806 06:46:59.003997 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.004031 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.004515 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.004783 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.004787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.005355 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:46:59.005523 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.005527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.006248 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.006249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.007059 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.007062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.007467 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.007468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.007602 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.007995 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:46:59.008003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:46:59.009381 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:46:59.010977 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:59.010984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:59.010986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:59.011203 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:59.011419 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:46:59.011420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:46:59.011420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:46:59.011521 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:46:59.011776 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:46:59.011783 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:46:59.011913 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.012022 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.012197 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260806 06:46:59.012580 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 17305
I20260806 06:46:59.012682 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:46:59.012887 6373 external_mini_cluster.cc:1428] Setting key 58355d07eb043a262da2f7cebbab4b06
I20260806 06:46:59.012959 17439 heartbeater.cc:347] Connected to a master server at 127.6.57.126:43791
I20260806 06:46:59.013051 17439 heartbeater.cc:464] Registering TS with master...
I20260806 06:46:59.013262 17439 heartbeater.cc:511] Master 127.6.57.126:43791 requested a full tablet report, sending...
I20260806 06:46:59.013645 16959 ts_manager.cc:194] Registered new tserver with Master: fa04231a0d354a8cbed3f7780e21e53e (127.6.57.67:41915)
I20260806 06:46:59.014034 16959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:34947
I20260806 06:46:59.024626 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:46:59.031735 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.025921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.6.57.126:43791)
0806 06:46:59.026039 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.026041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.026181 (+ 140us) server_negotiation.cc:407] Connection header received
0806 06:46:59.026209 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.026210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.026221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.026283 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:46:59.026909 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.027367 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.028106 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.028284 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.028336 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.029450 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:46:59.029451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:46:59.029451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:46:59.029457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:46:59.030854 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:59.031138 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:59.031139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:59.031139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:59.031188 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:46:59.031328 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:46:59.031328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:46:59.031328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:46:59.031448 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:46:59.031534 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.031596 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.031650 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 06:46:59.033304 16959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:46:59.034919 16959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:46:59.041677 17451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:37351 (local address 127.0.0.1:45176)
0806 06:46:59.037558 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.037600 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.037698 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.038169 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.038170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.038171 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:59.038236 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.038238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039589 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.039590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.040514 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.040517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041444 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.041508 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041519 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.041571 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.041605 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":2,"thread_start_us":89,"threads_started":1}
I20260806 06:46:59.041750 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45176 (local address 127.6.57.65:37351)
0806 06:46:59.037983 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.037988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.037999 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:46:59.038035 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.038037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.038051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.038318 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:59.038391 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039467 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.040628 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041359 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041445 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041507 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.041616 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.041659 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":73,"threads_started":1}
I20260806 06:46:59.041793 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.6.57.66:34911)
0806 06:46:59.038043 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.038048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.038058 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:46:59.038098 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.038101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.038117 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.038229 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:59.038342 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039113 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.040572 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041197 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041288 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041352 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.041654 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.041700 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260806 06:46:59.041591 17450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:34911 (local address 127.0.0.1:60370)
0806 06:46:59.037815 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.037823 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.037860 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.038229 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.038230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.038232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:59.038276 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.038278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039211 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.039213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.040467 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.040469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041287 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.041365 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041379 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.041447 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.041498 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":82,"threads_started":1}
I20260806 06:46:59.042513 17454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:41915 (local address 127.0.0.1:53106)
0806 06:46:59.037746 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.037757 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.037807 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.038154 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.038157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.038160 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:59.038236 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.038239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039561 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.039562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.040645 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.040647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041564 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.041629 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041633 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.041684 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.042420 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":10,"thread_start_us":110,"threads_started":1}
I20260806 06:46:59.042558 17091 tablet_service.cc:1467] Processing CreateTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb (DEFAULT_TABLE table=test-table [id=d09f33d3c247455983f13b9fbfcb348f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:59.042695 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.037493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.6.57.67:41915)
0806 06:46:59.037703 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.037709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.037897 (+ 188us) server_negotiation.cc:407] Connection header received
0806 06:46:59.037948 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.037951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.037968 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.038042 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:59.038321 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.039441 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.040717 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.041426 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.041483 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.041529 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.042264 (+ 735us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.042309 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":91,"threads_started":1}
I20260806 06:46:59.042577 17231 tablet_service.cc:1467] Processing CreateTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb (DEFAULT_TABLE table=test-table [id=d09f33d3c247455983f13b9fbfcb348f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:59.042958 17231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be24778e9b7e4cfc9abf2dfc569511cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:59.042956 17091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be24778e9b7e4cfc9abf2dfc569511cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:59.043315 17372 tablet_service.cc:1467] Processing CreateTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb (DEFAULT_TABLE table=test-table [id=d09f33d3c247455983f13b9fbfcb348f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:46:59.043625 17372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be24778e9b7e4cfc9abf2dfc569511cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:46:59.045588 17456 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Bootstrap starting.
I20260806 06:46:59.046293 17456 tablet_bootstrap.cc:654] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:59.046280 17457 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Bootstrap starting.
I20260806 06:46:59.047012 17456 log.cc:824] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:59.047057 17457 tablet_bootstrap.cc:654] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:59.047601 17457 log.cc:824] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:59.047835 17456 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: No bootstrap required, opened a new log
I20260806 06:46:59.047914 17456 ts_tablet_manager.cc:1397] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:59.047948 17458 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Bootstrap starting.
I20260806 06:46:59.048345 17457 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: No bootstrap required, opened a new log
I20260806 06:46:59.048430 17457 ts_tablet_manager.cc:1397] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:46:59.048581 17458 tablet_bootstrap.cc:654] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Neither blocks nor log segments found. Creating new log.
I20260806 06:46:59.049136 17458 log.cc:824] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:46:59.049928 17458 tablet_bootstrap.cc:492] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: No bootstrap required, opened a new log
I20260806 06:46:59.049995 17458 ts_tablet_manager.cc:1397] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:46:59.050125 17456 raft_consensus.cc:348] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.050284 17456 raft_consensus.cc:374] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:59.050174 17457 raft_consensus.cc:348] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.050324 17456 raft_consensus.cc:729] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94475ad2714343ae950db63bbfb1c5d0, State: Initialized, Role: FOLLOWER
I20260806 06:46:59.050335 17457 raft_consensus.cc:374] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:59.050371 17457 raft_consensus.cc:729] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4171d65aabda4e3c96c1a31204592b8a, State: Initialized, Role: FOLLOWER
I20260806 06:46:59.050457 17456 consensus_queue.cc:260] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.050536 17457 consensus_queue.cc:260] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.050756 17456 ts_tablet_manager.cc:1428] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:59.050827 17457 ts_tablet_manager.cc:1428] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:46:59.050881 17300 heartbeater.cc:503] Master 127.6.57.126:43791 was elected leader, sending a full tablet report...
I20260806 06:46:59.050943 17160 heartbeater.cc:503] Master 127.6.57.126:43791 was elected leader, sending a full tablet report...
I20260806 06:46:59.051976 17458 raft_consensus.cc:348] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.052167 17458 raft_consensus.cc:374] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:46:59.052222 17458 raft_consensus.cc:729] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa04231a0d354a8cbed3f7780e21e53e, State: Initialized, Role: FOLLOWER
I20260806 06:46:59.052328 17458 consensus_queue.cc:260] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.052632 17458 ts_tablet_manager.cc:1428] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:46:59.052708 17439 heartbeater.cc:503] Master 127.6.57.126:43791 was elected leader, sending a full tablet report...
W20260806 06:46:59.089893 17301 tablet.cc:2369] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:46:59.177809 17161 tablet.cc:2369] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:59.250077 17465 raft_consensus.cc:482] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:46:59.250216 17465 raft_consensus.cc:504] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.250708 17465 leader_election.cc:288] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4171d65aabda4e3c96c1a31204592b8a (127.6.57.65:37351), 94475ad2714343ae950db63bbfb1c5d0 (127.6.57.66:34911)
W20260806 06:46:59.253669 17440 tablet.cc:2369] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:46:59.253753 17452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.250998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:45519 (local address 127.6.57.65:37351)
0806 06:46:59.251075 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.251079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.251091 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:46:59.251127 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.251128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.251141 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.251255 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:59.251444 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.251984 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.252988 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.253384 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.253436 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.253445 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.253617 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.253657 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:46:59.253816 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.250928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:37351 (local address 127.6.57.67:45519)
0806 06:46:59.250986 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.251000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.251048 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.251263 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.251265 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.251267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:59.251345 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.251347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.252058 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.252059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.252902 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.252904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.253451 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.253452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.253548 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.253552 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.253601 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.253640 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:46:59.253878 17455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.251140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:33173 (local address 127.6.57.66:34911)
0806 06:46:59.251213 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.251215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.251353 (+ 138us) server_negotiation.cc:407] Connection header received
0806 06:46:59.251382 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.251383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.251397 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.251456 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:46:59.251594 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.252081 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.253038 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.253466 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.253535 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.253546 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.253751 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.253807 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:46:59.254101 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.251070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:34911 (local address 127.6.57.67:33173)
0806 06:46:59.251287 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.251295 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:46:59.251343 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:46:59.251464 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:46:59.251464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:46:59.251466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:46:59.251530 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.251531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.252162 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.252163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.252927 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:46:59.252929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.253557 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.253559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:46:59.253641 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.253645 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:46:59.253967 (+ 322us) client_negotiation.cc:240] Negotiation successful
0806 06:46:59.254012 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":145,"threads_started":1}
I20260806 06:46:59.254338 17111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be24778e9b7e4cfc9abf2dfc569511cb" candidate_uuid: "fa04231a0d354a8cbed3f7780e21e53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4171d65aabda4e3c96c1a31204592b8a" is_pre_election: true
I20260806 06:46:59.254532 17111 raft_consensus.cc:2454] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa04231a0d354a8cbed3f7780e21e53e in term 0.
I20260806 06:46:59.254688 17251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be24778e9b7e4cfc9abf2dfc569511cb" candidate_uuid: "fa04231a0d354a8cbed3f7780e21e53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94475ad2714343ae950db63bbfb1c5d0" is_pre_election: true
I20260806 06:46:59.254732 17324 leader_election.cc:302] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4171d65aabda4e3c96c1a31204592b8a, fa04231a0d354a8cbed3f7780e21e53e; no voters:
I20260806 06:46:59.254842 17251 raft_consensus.cc:2454] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa04231a0d354a8cbed3f7780e21e53e in term 0.
I20260806 06:46:59.254833 17465 raft_consensus.cc:2781] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:46:59.254911 17465 raft_consensus.cc:482] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:46:59.254933 17465 raft_consensus.cc:3037] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:59.255729 17465 raft_consensus.cc:504] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.255859 17465 leader_election.cc:288] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [CANDIDATE]: Term 1 election: Requested vote from peers 4171d65aabda4e3c96c1a31204592b8a (127.6.57.65:37351), 94475ad2714343ae950db63bbfb1c5d0 (127.6.57.66:34911)
I20260806 06:46:59.256062 17251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be24778e9b7e4cfc9abf2dfc569511cb" candidate_uuid: "fa04231a0d354a8cbed3f7780e21e53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94475ad2714343ae950db63bbfb1c5d0"
I20260806 06:46:59.256141 17251 raft_consensus.cc:3037] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:59.256122 17111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be24778e9b7e4cfc9abf2dfc569511cb" candidate_uuid: "fa04231a0d354a8cbed3f7780e21e53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4171d65aabda4e3c96c1a31204592b8a"
I20260806 06:46:59.256232 17111 raft_consensus.cc:3037] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:46:59.256911 17251 raft_consensus.cc:2454] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa04231a0d354a8cbed3f7780e21e53e in term 1.
I20260806 06:46:59.256930 17111 raft_consensus.cc:2454] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa04231a0d354a8cbed3f7780e21e53e in term 1.
I20260806 06:46:59.257084 17324 leader_election.cc:302] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94475ad2714343ae950db63bbfb1c5d0, fa04231a0d354a8cbed3f7780e21e53e; no voters:
I20260806 06:46:59.257215 17465 raft_consensus.cc:2781] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:46:59.257341 17465 raft_consensus.cc:686] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 1 LEADER]: Becoming Leader. State: Replica: fa04231a0d354a8cbed3f7780e21e53e, State: Running, Role: LEADER
I20260806 06:46:59.257436 17465 consensus_queue.cc:237] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } }
I20260806 06:46:59.258404 16959 catalog_manager.cc:5495] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e reported cstate change: term changed from 0 to 1, leader changed from <none> to fa04231a0d354a8cbed3f7780e21e53e (127.6.57.67). New cstate: current_term: 1 leader_uuid: "fa04231a0d354a8cbed3f7780e21e53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa04231a0d354a8cbed3f7780e21e53e" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 41915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 } health_report { overall_health: UNKNOWN } } }
I20260806 06:46:59.268697 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.266186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.6.57.67:41915)
0806 06:46:59.266248 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:46:59.266250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:46:59.266274 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:46:59.266347 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:46:59.266348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:46:59.266366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:46:59.266423 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:46:59.266570 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.267145 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.267883 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:46:59.268107 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:46:59.268183 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:46:59.268346 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:46:59.268456 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:46:59.268557 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:46:59.268582 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 06:46:59.268601 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:46:59.270336 17111 raft_consensus.cc:1261] T be24778e9b7e4cfc9abf2dfc569511cb P 4171d65aabda4e3c96c1a31204592b8a [term 1 FOLLOWER]: Refusing update from remote peer fa04231a0d354a8cbed3f7780e21e53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:59.270351 17251 raft_consensus.cc:1261] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 1 FOLLOWER]: Refusing update from remote peer fa04231a0d354a8cbed3f7780e21e53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:46:59.270701 17467 consensus_queue.cc:1035] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4171d65aabda4e3c96c1a31204592b8a" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 37351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:59.270812 17465 consensus_queue.cc:1035] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "94475ad2714343ae950db63bbfb1c5d0" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:46:59.273052 17470 mvcc.cc:205] Tried to move back new op lower bound from 7315451163728740352 to 7315451163679330304. Current Snapshot: MvccSnapshot[applied={T|T < 7315451163728740352}]
I20260806 06:46:59.273413 17471 mvcc.cc:205] Tried to move back new op lower bound from 7315451163728740352 to 7315451163679330304. Current Snapshot: MvccSnapshot[applied={T|T < 7315451163728740352}]
I20260806 06:46:59.275851 16960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:59.275936 16960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:46:59.276687 16960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf03a65f78b44d318f10978587c20ce1: Sending DeleteTablet for 3 replicas of tablet be24778e9b7e4cfc9abf2dfc569511cb
I20260806 06:46:59.277083 17091 tablet_service.cc:1514] Processing DeleteTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:45176
I20260806 06:46:59.277073 17372 tablet_service.cc:1514] Processing DeleteTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53106
I20260806 06:46:59.277093 17231 tablet_service.cc:1514] Processing DeleteTablet for tablet be24778e9b7e4cfc9abf2dfc569511cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:46:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:60370
I20260806 06:46:59.277329 17481 tablet_replica.cc:330] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: stopping tablet replica
I20260806 06:46:59.277437 17481 raft_consensus.cc:2229] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:46:59.277617 17481 raft_consensus.cc:2258] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:59.277688 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 17026
I20260806 06:46:59.281522 17481 ts_tablet_manager.cc:1905] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:59.282618 17479 tablet_replica.cc:330] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: stopping tablet replica
I20260806 06:46:59.282784 17479 raft_consensus.cc:2229] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:46:59.282953 17479 raft_consensus.cc:2258] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:46:59.283360 17479 ts_tablet_manager.cc:1905] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:46:59.283689 17481 ts_tablet_manager.cc:1918] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:59.283772 17481 log.cc:1196] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/be24778e9b7e4cfc9abf2dfc569511cb
I20260806 06:46:59.284082 17481 ts_tablet_manager.cc:1939] T be24778e9b7e4cfc9abf2dfc569511cb P 94475ad2714343ae950db63bbfb1c5d0: Deleting consensus metadata
I20260806 06:46:59.284740 16946 catalog_manager.cc:4841] TS 94475ad2714343ae950db63bbfb1c5d0 (127.6.57.66:34911): tablet be24778e9b7e4cfc9abf2dfc569511cb (table test-table [id=d09f33d3c247455983f13b9fbfcb348f]) successfully deleted
I20260806 06:46:59.285355 17479 ts_tablet_manager.cc:1918] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:46:59.285429 17479 log.cc:1196] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/be24778e9b7e4cfc9abf2dfc569511cb
I20260806 06:46:59.285748 17479 ts_tablet_manager.cc:1939] T be24778e9b7e4cfc9abf2dfc569511cb P fa04231a0d354a8cbed3f7780e21e53e: Deleting consensus metadata
W20260806 06:46:59.286149 16946 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:37351 (error 108)
I20260806 06:46:59.286231 16945 catalog_manager.cc:4841] TS fa04231a0d354a8cbed3f7780e21e53e (127.6.57.67:41915): tablet be24778e9b7e4cfc9abf2dfc569511cb (table test-table [id=d09f33d3c247455983f13b9fbfcb348f]) successfully deleted
I20260806 06:46:59.286377 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 17166
I20260806 06:46:59.287298 17454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:46:59.287166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:37351 (local address 127.0.0.1:45178)
0806 06:46:59.287212 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:46:59.287263 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:37351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 06:46:59.287420 16946 catalog_manager.cc:4584] TS 4171d65aabda4e3c96c1a31204592b8a (127.6.57.65:37351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be24778e9b7e4cfc9abf2dfc569511cb: Network error: Client connection negotiation failed: client connection to 127.6.57.65:37351: connect: Connection refused (error 111)
I20260806 06:46:59.294004 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 17305
I20260806 06:46:59.299485 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 16928
2026-08-06T06:46:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2142 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:46:59 dist-test-slave-5l3k krb5kdc[17485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:46:59 dist-test-slave-5l3k krb5kdc[17485](info): set up 2 sockets
Aug 06 06:46:59 dist-test-slave-5l3k krb5kdc[17485](info): commencing operation
krb5kdc: starting...
W20260806 06:47:00.349308 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:47:00 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:47:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangeradmin_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangeradmin_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangerlookup_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangerlookup_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
I20260806 06:47:00.510941 6373 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXFWrbB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/postgres -l logfile start
2026-08-06 06:47:03.017 UTC [17517] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:47:03.017 UTC [17517] LOG: listening on IPv4 address "127.6.57.84", port 45239
2026-08-06 06:47:03.020 UTC [17517] LOG: listening on Unix socket "/tmp/.s.PGSQL.45239"
2026-08-06 06:47:03.055 UTC [17521] LOG: database system was shut down at 2026-08-06 06:47:01 UTC
2026-08-06 06:47:03.060 UTC [17517] LOG: database system is ready to accept connections
I20260806 06:47:04.017120 6373 mini_postgres.cc:96] Postgres bound to 45239
2026-08-06 06:47:04.026 UTC [17529] FATAL: database "slave" does not exist
127.6.57.84:45239 - accepting connections
I20260806 06:47:04.026888 6373 mini_ranger.cc:161] Starting Ranger...
I20260806 06:47:04.043080 6373 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:47:04.497828 6373 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:47:04.497896 6373 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin
2026-08-06 06:47:04,820 [I] DB FLAVOR :POSTGRES
2026-08-06 06:47:04,821 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:47:04,821 [I] Checking connection..
2026-08-06 06:47:04,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:47:05,080 [I] Checking connection passed.
2026-08-06 06:47:05,080 [I] --------- Verifying version history table ---------
2026-08-06 06:47:05,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:05,337 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:47:05,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:05,595 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:47:05,596 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:47:05,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:47:05.844 UTC [17594] WARNING: there is no transaction in progress
2026-08-06 06:47:05,850 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:47:05,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:06,108 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:47:06,109 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:47:06,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:47:06,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:47:06,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5l3k.c.gcp-upstream.internal','N') ;"
2026-08-06 06:47:06,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:47:07,100 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:47:07,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:47:07,356 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:47:07,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:47:07,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:47:07,832 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:47:07,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:47:08.786 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.802 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.818 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.833 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.851 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.872 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.929 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.936 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.949 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.959 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.969 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.982 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.991 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:08.997 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:09.004 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:09.010 UTC [17709] WARNING: there is no transaction in progress
2026-08-06 06:47:09,650 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:47:09,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:47:09,887 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:47:09,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:47:10,139 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:47:10,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:47:10,381 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:47:10,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:47:10,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:47:10,888 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5l3k.c.gcp-upstream.internal';"
2026-08-06 06:47:11,133 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:47:11,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:47:11,375 [I] DB_PATCHES have already been applied
I20260806 06:47:11.381040 6373 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXFWrbB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:47:11.381122 6373 mini_ranger.cc:193] Using host: 127.6.57.84
I20260806 06:47:11.384815 6373 mini_ranger.cc:227] Ranger admin URL: http://127.6.57.84:57447
Aug 06, 2026 6:47:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:47:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:47:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:47:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:47:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:47:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:47:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:47:12.508636 6373 mini_ranger.cc:160] Time spent starting Ranger: real 8.482s user 0.005s sys 0.006s
Aug 06, 2026 6:47:32 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:47:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 6:47:33 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:47:33 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:47:34 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:47:34.587852 6373 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangerkms_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/rangerkms_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.57.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.57.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:47:34 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998854, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:47:34.675575 6373 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:47:34.685518 6373 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:47:35.534088 6373 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:47:35.534176 6373 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms
I20260806 06:47:35.534197 6373 mini_ranger_kms.cc:227] Using postgres at 127.6.57.84:45239
2026-08-06 06:47:35,904 [I] DB FLAVOR :POSTGRES
2026-08-06 06:47:35,904 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:47:35,904 [I] Checking connection
2026-08-06 06:47:35,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:47:36,138 [I] connection success
2026-08-06 06:47:36,138 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:47:36,138 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:47:36,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:47:36,370 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:47:36,371 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:47:36,371 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:47:36,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:45239/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:47:36,644 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:47:37.098773 6373 mini_ranger_kms.cc:310] Created kms service
I20260806 06:47:37.307840 6373 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:47:37.352594 6373 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:47:37 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998854, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:47:37.766968 6373 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:47:37.767087 6373 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:47:37.767112 6373 mini_ranger_kms.cc:240] Using host: 127.6.57.84
I20260806 06:47:37.771257 6373 mini_ranger_kms.cc:276] Ranger KMS PID: 17898
I20260806 06:47:37.771376 6373 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.57.84:33679
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:47:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 06:47:40.077389 6373 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.402s user 0.015s sys 0.259s
I20260806 06:47:40.077499 6373 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:47:40.077593 6373 mini_ranger_kms.cc:401] 127.6.57.84:33679/kms/v1/keys
Aug 06 06:47:40 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:47:40 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
Aug 06 06:47:40 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998860, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
Aug 06 06:47:43 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:47:43.164640 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36585
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:36585
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:33679/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:47:43.247704 17950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:47:43.247916 17950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:47:43.247938 17950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:47:43.249110 17950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:47:43.249151 17950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:47:43.249178 17950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:47:43.249191 17950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:47:43.249202 17950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:47:43.251374 17950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.57.84:33679/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:36585
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:47:43.251653 17950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:47:43.251878 17950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:47:43.257016 17950 server_base.cc:1034] running on GCE node
W20260806 06:47:43.256943 17956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:43.256943 17958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:43.256955 17955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:47:43.257524 17950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:47:43.257835 17950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:47:43.258980 17950 hybrid_clock.cc:648] HybridClock initialized: now 1785998863258967 us; error 32 us; skew 500 ppm
Aug 06 06:47:43 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:47:43.266330 17950 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:47:43.266831 17950 webserver.cc:460] Webserver started at http://127.6.57.126:45839/ using document root <none> and password file <none>
I20260806 06:47:43.267081 17950 fs_manager.cc:359] Metadata directory not provided
I20260806 06:47:43.267144 17950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:47:43.267328 17950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:47:43 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:47:43.321995 17950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "35c88e984c7642458d30ef6f8a091731"
format_stamp: "Formatted at 2026-08-06 06:47:43 on dist-test-slave-5l3k"
server_key: "0ad82adaa4565537238f94cafe5b0a05"
server_key_iv: "e52557fe66790226de595076567b61c8"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:43.322459 17950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "35c88e984c7642458d30ef6f8a091731"
format_stamp: "Formatted at 2026-08-06 06:47:43 on dist-test-slave-5l3k"
server_key: "0ad82adaa4565537238f94cafe5b0a05"
server_key_iv: "e52557fe66790226de595076567b61c8"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:43.324744 17950 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 06:47:43.336212 17966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:43.336400 17950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:47:43.336479 17950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "35c88e984c7642458d30ef6f8a091731"
format_stamp: "Formatted at 2026-08-06 06:47:43 on dist-test-slave-5l3k"
server_key: "0ad82adaa4565537238f94cafe5b0a05"
server_key_iv: "e52557fe66790226de595076567b61c8"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:43.336553 17950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:47:43.353859 17950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:47:43.355967 17950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:47:43.356122 17950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:47:43.361481 17950 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:36585
I20260806 06:47:43.361578 18020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:36585 every 8 connection(s)
I20260806 06:47:43.361929 17950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:47:43.362818 18021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:47:43.365130 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: Bootstrap starting.
I20260806 06:47:43.365729 18021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: Neither blocks nor log segments found. Creating new log.
I20260806 06:47:43.366030 18021 log.cc:824] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: Log is configured to *not* fsync() on all Append() calls
I20260806 06:47:43.366853 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: No bootstrap required, opened a new log
I20260806 06:47:43.368873 18021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } }
I20260806 06:47:43.369012 18021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:47:43.369035 18021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c88e984c7642458d30ef6f8a091731, State: Initialized, Role: FOLLOWER
I20260806 06:47:43.369177 18021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } }
I20260806 06:47:43.369248 18021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:47:43.369274 18021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:47:43.369315 18021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:47:43.370085 18021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } }
I20260806 06:47:43.370214 18021 leader_election.cc:302] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c88e984c7642458d30ef6f8a091731; no voters:
I20260806 06:47:43.370374 18021 leader_election.cc:288] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:47:43.370674 18021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:47:43.370682 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 17950
I20260806 06:47:43.370781 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:47:43.371011 6373 external_mini_cluster.cc:1428] Setting key 20f200f08e7c7f1d09a5bee0d471202f
I20260806 06:47:43.371168 18024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:47:43.371775 18024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [term 1 LEADER]: Becoming Leader. State: Replica: 35c88e984c7642458d30ef6f8a091731, State: Running, Role: LEADER
I20260806 06:47:43.371925 18024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } }
I20260806 06:47:43.372558 18025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35c88e984c7642458d30ef6f8a091731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } } }
I20260806 06:47:43.372637 18025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [sys.catalog]: This master's current role is: LEADER
I20260806 06:47:43.372756 18025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c88e984c7642458d30ef6f8a091731. Latest consensus state: current_term: 1 leader_uuid: "35c88e984c7642458d30ef6f8a091731" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c88e984c7642458d30ef6f8a091731" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36585 } } }
I20260806 06:47:43.372797 18025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731 [sys.catalog]: This master's current role is: LEADER
I20260806 06:47:43.372798 18021 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:47:43.373955 18040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:47:43.374459 18040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:47:43.380607 18040 catalog_manager.cc:1347] Generated new cluster ID: c817ccf2418c4f81908feb82c19f40b9
I20260806 06:47:43.380748 18040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:47:43 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:47:43.386772 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:43.371516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.6.57.126:36585)
0806 06:47:43.372540 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:43.372550 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:43.372585 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:47:43.372740 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:43.372748 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:43.372833 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:43.373000 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:47:43.373950 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:43.377071 (+ 3121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:43.377945 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:43.378987 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:43.379114 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:43.381933 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:47:43.381938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:47:43.381938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:47:43.381950 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:47:43.384721 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:43.385113 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:43.385116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:43.385116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:43.385238 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:43.385440 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:43.385441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:43.385442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:43.385745 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:47:43.385902 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:43.386426 (+ 524us) server_negotiation.cc:299] Negotiation successful
0806 06:47:43.386569 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":102,"threads_started":1}
I20260806 06:47:43.398162 18040 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:47:43.398883 18040 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:47:43.405498 18040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: Generated new TSK 0
I20260806 06:47:43.405681 18040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:47:44.153383 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:33679/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:47:44.244099 18066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:47:44.244328 18066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:47:44.244367 18066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:47:44.245517 18066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:47:44.245582 18066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:47:44.245625 18066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:47:44.247150 18066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.57.84:33679/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:47:44.247419 18066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:47:44.247653 18066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:47:44.252666 18075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.252780 18072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.252877 18073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:47:44.253355 18066 server_base.cc:1034] running on GCE node
I20260806 06:47:44.253551 18066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:47:44.253822 18066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:47:44.255005 18066 hybrid_clock.cc:648] HybridClock initialized: now 1785998864254989 us; error 32 us; skew 500 ppm
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:47:44.257897 18066 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:47:44.258378 18066 webserver.cc:460] Webserver started at http://127.6.57.65:35793/ using document root <none> and password file <none>
I20260806 06:47:44.258657 18066 fs_manager.cc:359] Metadata directory not provided
I20260806 06:47:44.258730 18066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:47:44.258864 18066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:47:44.272877 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "69be8514909a42f08f736f6309b55aa3"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a2b40f3e2a31796d1ee32cd2596c3ae3"
server_key_iv: "c805661eba74b4b9dc8f89025057846d"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.273334 18066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "69be8514909a42f08f736f6309b55aa3"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a2b40f3e2a31796d1ee32cd2596c3ae3"
server_key_iv: "c805661eba74b4b9dc8f89025057846d"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.279229 18066 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.003s
I20260806 06:47:44.292474 18083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.292749 18066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:47:44.292862 18066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "69be8514909a42f08f736f6309b55aa3"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a2b40f3e2a31796d1ee32cd2596c3ae3"
server_key_iv: "c805661eba74b4b9dc8f89025057846d"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.292953 18066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:47:44.308667 18066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:47:44.311250 18066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:47:44.311409 18066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:47:44.311652 18066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:47:44.312076 18066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:47:44.312150 18066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.312181 18066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:47:44.312235 18066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.319540 18066 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:40497
I20260806 06:47:44.320034 18066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:47:44.322059 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 18066
I20260806 06:47:44.322176 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:47:44.322374 6373 external_mini_cluster.cc:1428] Setting key 889e2514001b534734c906f8734610c9
I20260806 06:47:44.321424 18201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:40497 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:47:44.344532 18205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.322856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36585 (local address 127.6.57.65:57013)
0806 06:47:44.326584 (+ 3728us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.326633 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.327350 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.334654 (+ 7304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.334659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.334928 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:47:44.335207 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.335214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.336288 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.336289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.337525 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.337528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.338071 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.338072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.338225 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.338873 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:47:44.338883 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:47:44.340552 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:47:44.342802 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.342809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.342811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.343112 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.343367 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.343368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.343368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.343499 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.343833 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:47:44.343841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:47:44.344015 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.344112 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.344263 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3678,"thread_start_us":84,"threads_started":1}
I20260806 06:47:44.345320 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.322877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:57013 (local address 127.6.57.126:36585)
0806 06:47:44.334390 (+ 11513us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.334393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.334404 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:47:44.334434 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.334435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.334449 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.334556 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:47:44.335357 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.336172 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.337639 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.337983 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.338046 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.340664 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:47:44.340666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:47:44.340666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:47:44.340675 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:47:44.342696 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.343213 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.343214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.343214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.343280 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.343599 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.343600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.343601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.343744 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:47:44.345092 (+ 1348us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.345135 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.345228 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11476,"thread_start_us":89,"threads_started":1}
I20260806 06:47:44.345885 18202 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36585
I20260806 06:47:44.345976 18202 heartbeater.cc:464] Registering TS with master...
I20260806 06:47:44.346163 18202 heartbeater.cc:511] Master 127.6.57.126:36585 requested a full tablet report, sending...
I20260806 06:47:44.346815 17984 ts_manager.cc:194] Registered new tserver with Master: 69be8514909a42f08f736f6309b55aa3 (127.6.57.65:40497)
I20260806 06:47:44.347683 17984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:57013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:47:44.357360 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:33679/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:47:44.464311 18209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:47:44.464529 18209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:47:44.464567 18209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:47:44.465719 18209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:47:44.465786 18209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:47:44.465832 18209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:47:44.467537 18209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.57.84:33679/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:47:44.467793 18209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:47:44.468031 18209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:47:44.473459 18217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.473543 18219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.473587 18216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:47:44.474339 18209 server_base.cc:1034] running on GCE node
I20260806 06:47:44.474583 18209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:47:44.474903 18209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:47:44.476081 18209 hybrid_clock.cc:648] HybridClock initialized: now 1785998864476064 us; error 45 us; skew 500 ppm
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:47:44.479101 18209 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:47:44.479599 18209 webserver.cc:460] Webserver started at http://127.6.57.66:39129/ using document root <none> and password file <none>
I20260806 06:47:44.479857 18209 fs_manager.cc:359] Metadata directory not provided
I20260806 06:47:44.479924 18209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:47:44.480057 18209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:47:44.496878 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "bc59cd1e2cc4454eb133f482c90df099"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a71c4ef16bb395efb0640dea6dfbb059"
server_key_iv: "3cb3a679880db8578d7f89a7bc6014dd"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.497690 18209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "bc59cd1e2cc4454eb133f482c90df099"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a71c4ef16bb395efb0640dea6dfbb059"
server_key_iv: "3cb3a679880db8578d7f89a7bc6014dd"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.499902 18209 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 06:47:44.517483 18227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.517745 18209 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 06:47:44.517836 18209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "bc59cd1e2cc4454eb133f482c90df099"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "a71c4ef16bb395efb0640dea6dfbb059"
server_key_iv: "3cb3a679880db8578d7f89a7bc6014dd"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.517918 18209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:47:44.529203 18209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:47:44.539727 18209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:47:44.539865 18209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:47:44.540035 18209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:47:44.540424 18209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:47:44.540465 18209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.540493 18209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:47:44.540527 18209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.548120 18209 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:33591
I20260806 06:47:44.548552 18209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:47:44.549384 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:33591 every 8 connection(s)
I20260806 06:47:44.553216 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 18209
I20260806 06:47:44.553314 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:47:44.553503 6373 external_mini_cluster.cc:1428] Setting key 8d3664db4199bfc59a4e27c047d19a73
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:47:44.560336 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.550109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37759 (local address 127.6.57.126:36585)
0806 06:47:44.550159 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.550161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.550402 (+ 241us) server_negotiation.cc:407] Connection header received
0806 06:47:44.551056 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.551058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.551071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.551135 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:47:44.551817 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.552609 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.554079 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.554385 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.554426 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.557053 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:47:44.557055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:47:44.557055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:47:44.557062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:47:44.558789 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.559245 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.559246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.559246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.559309 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.559594 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.559595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.559596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.559738 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:47:44.559812 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.560146 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.560237 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:47:44.560490 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.550070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36585 (local address 127.6.57.66:37759)
0806 06:47:44.550234 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.550281 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.550950 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.551202 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.551206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.551438 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:47:44.551726 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.551732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.552714 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.552716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.553972 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.553975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.554577 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.554578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.554731 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.555307 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:47:44.555317 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:47:44.556957 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:47:44.558872 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.558879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.558881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.559162 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.559384 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.559385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.559386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.559515 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.559815 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:47:44.559821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:47:44.560000 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.560116 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.560263 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260806 06:47:44.561141 18346 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36585
I20260806 06:47:44.561237 18346 heartbeater.cc:464] Registering TS with master...
I20260806 06:47:44.561440 18346 heartbeater.cc:511] Master 127.6.57.126:36585 requested a full tablet report, sending...
I20260806 06:47:44.561825 17984 ts_manager.cc:194] Registered new tserver with Master: bc59cd1e2cc4454eb133f482c90df099 (127.6.57.66:33591)
I20260806 06:47:44.562299 17984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:37759
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:47:44.600572 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:33679/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:47:44.726577 18352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:47:44.726840 18352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:47:44.726876 18352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:47:44.728572 18352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:47:44.728632 18352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:47:44.728673 18352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:47:44.730825 18352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.57.84:33679/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:36585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:47:44.731087 18352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:47:44.731400 18352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:47:44.737509 18358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.737515 18357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:47:44.738108 18360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:47:44.738770 18352 server_base.cc:1034] running on GCE node
I20260806 06:47:44.738974 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:47:44.739270 18352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:47:44.740444 18352 hybrid_clock.cc:648] HybridClock initialized: now 1785998864740421 us; error 37 us; skew 500 ppm
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:47:44.743592 18352 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:47:44.744150 18352 webserver.cc:460] Webserver started at http://127.6.57.67:45237/ using document root <none> and password file <none>
I20260806 06:47:44.744407 18352 fs_manager.cc:359] Metadata directory not provided
I20260806 06:47:44.744478 18352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:47:44.744683 18352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:47:44.757450 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "0a4141a9738d4833a7855c1e7bf223ce"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "dd7ec2dd820b8f5f3213568ded45ef40"
server_key_iv: "866f0071f07a36fa3c838bdc74bfa674"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.757968 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "0a4141a9738d4833a7855c1e7bf223ce"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "dd7ec2dd820b8f5f3213568ded45ef40"
server_key_iv: "866f0071f07a36fa3c838bdc74bfa674"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.762090 18352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 06:47:44.779994 18367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.780180 18352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:47:44.780270 18352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "0a4141a9738d4833a7855c1e7bf223ce"
format_stamp: "Formatted at 2026-08-06 06:47:44 on dist-test-slave-5l3k"
server_key: "dd7ec2dd820b8f5f3213568ded45ef40"
server_key_iv: "866f0071f07a36fa3c838bdc74bfa674"
server_key_version: "kuduclusterkey@0"
I20260806 06:47:44.780375 18352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:47:44.800634 18352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:47:44.803040 18352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:47:44.803215 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:47:44.803539 18352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:47:44.803977 18352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:47:44.804034 18352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.804075 18352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:47:44.804105 18352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:47:44.810854 18352 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:43747
I20260806 06:47:44.811324 18352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:47:44.812121 18483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:43747 every 8 connection(s)
I20260806 06:47:44.820662 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 18352
I20260806 06:47:44.820760 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:47:44.820948 6373 external_mini_cluster.cc:1428] Setting key f754e8f7a821a57518397ca7c76fc56a
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:47:44.837621 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.813010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:42821 (local address 127.6.57.126:36585)
0806 06:47:44.813056 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.813058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.813359 (+ 301us) server_negotiation.cc:407] Connection header received
0806 06:47:44.814111 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.814112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.814125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.814158 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:47:44.814986 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.815690 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.831728 (+ 16038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.832042 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.832125 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.834632 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:47:44.834634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:47:44.834634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:47:44.834642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:47:44.836207 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.836630 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.836631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.836632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.836700 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.836950 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.836951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.836951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.837069 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:47:44.837160 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.837452 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.837533 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:47:44.837936 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.812672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36585 (local address 127.6.57.67:42821)
0806 06:47:44.813239 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.813281 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.813958 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.814328 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.814332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.814590 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:47:44.814872 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.814879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.815828 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.815829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.831597 (+ 15768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.831602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.832203 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.832204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.832367 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.833034 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:47:44.833046 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:47:44.834466 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:47:44.836310 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.836317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.836320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.836544 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.836765 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:47:44.836766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:47:44.836766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:47:44.836877 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:47:44.837154 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:47:44.837161 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:47:44.837315 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.837417 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.837565 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":82,"threads_started":1}
I20260806 06:47:44.838436 18484 heartbeater.cc:347] Connected to a master server at 127.6.57.126:36585
I20260806 06:47:44.838553 18484 heartbeater.cc:464] Registering TS with master...
I20260806 06:47:44.838758 18484 heartbeater.cc:511] Master 127.6.57.126:36585 requested a full tablet report, sending...
I20260806 06:47:44.839166 17984 ts_manager.cc:194] Registered new tserver with Master: 0a4141a9738d4833a7855c1e7bf223ce (127.6.57.67:43747)
I20260806 06:47:44.839550 17984 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:42821
I20260806 06:47:44.843207 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:47:44.856323 18206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.844880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.6.57.126:36585)
0806 06:47:44.844915 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.844917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.845302 (+ 385us) server_negotiation.cc:407] Connection header received
0806 06:47:44.845399 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.845400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.845409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.845437 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:47:44.846330 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.847075 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.847898 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.848197 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.848286 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.850044 (+ 1758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:47:44.850046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:47:44.850046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:47:44.850053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:47:44.852422 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.854660 (+ 2238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.854661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.854662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.854725 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:47:44.855238 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:47:44.855239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:47:44.855239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:47:44.855416 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:47:44.855529 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.855942 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.856066 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:47:44.858282 17984 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:47:44.860636 17984 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:47:44.869287 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.863509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:40497 (local address 127.0.0.1:38566)
0806 06:47:44.864502 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.864543 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.864679 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.865016 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.865019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.865020 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:47:44.865094 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.865096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.866680 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.866681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.867927 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.867929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.868935 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.868937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.869041 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.869057 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.869146 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.869202 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260806 06:47:44.869845 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.863509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33591 (local address 127.0.0.1:37778)
0806 06:47:44.864876 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.864886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.864930 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.865204 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.865204 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.865206 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:47:44.865249 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.865251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.866738 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.866738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.867927 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.867930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.869287 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.869288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.869674 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.869678 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.869737 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.869780 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1335,"mutex_wait_us":159,"thread_start_us":90,"threads_started":1}
I20260806 06:47:44.869982 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.864251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38566 (local address 127.6.57.65:40497)
0806 06:47:44.864387 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.864391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.864600 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:47:44.864831 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.864834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.864853 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.864933 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:47:44.865188 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.866584 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.868026 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.868782 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.868845 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.869836 (+ 991us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.869865 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.869911 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":91,"threads_started":1}
I20260806 06:47:44.870433 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.864022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.6.57.66:33591)
0806 06:47:44.864210 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.864216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.864990 (+ 774us) server_negotiation.cc:407] Connection header received
0806 06:47:44.865036 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.865038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.865052 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.865125 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:47:44.865320 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.866330 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.868047 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.869123 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.869181 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.869255 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.870129 (+ 874us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.870360 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260806 06:47:44.870870 18116 tablet_service.cc:1467] Processing CreateTablet for tablet fc62be58bc694b228ac6048204296668 (DEFAULT_TABLE table=test-table [id=b86db676ca584486aa46fbfb7e63b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:47:44.871275 18116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62be58bc694b228ac6048204296668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:47:44.871189 18263 tablet_service.cc:1467] Processing CreateTablet for tablet fc62be58bc694b228ac6048204296668 (DEFAULT_TABLE table=test-table [id=b86db676ca584486aa46fbfb7e63b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:47:44.871506 18263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62be58bc694b228ac6048204296668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:47:44.875236 18503 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Bootstrap starting.
I20260806 06:47:44.875950 18503 tablet_bootstrap.cc:654] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Neither blocks nor log segments found. Creating new log.
I20260806 06:47:44.876626 18504 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Bootstrap starting.
I20260806 06:47:44.877533 18504 tablet_bootstrap.cc:654] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Neither blocks nor log segments found. Creating new log.
I20260806 06:47:44.878430 18503 log.cc:824] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:47:44.877883 18504 log.cc:824] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Log is configured to *not* fsync() on all Append() calls
I20260806 06:47:44.879511 18503 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: No bootstrap required, opened a new log
I20260806 06:47:44.879583 18503 ts_tablet_manager.cc:1397] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 06:47:44.882582 18503 raft_consensus.cc:348] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.882772 18503 raft_consensus.cc:374] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:47:44.882817 18503 raft_consensus.cc:729] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69be8514909a42f08f736f6309b55aa3, State: Initialized, Role: FOLLOWER
I20260806 06:47:44.882737 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.863733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43747 (local address 127.0.0.1:34404)
0806 06:47:44.866872 (+ 3139us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:44.866881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:44.866925 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:44.878360 (+ 11435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:44.878361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:44.878363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:47:44.878401 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.878402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.879981 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.879982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.881161 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:44.881165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.882528 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.882529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:44.882590 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.882595 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:47:44.882631 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 06:47:44.882669 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3117,"thread_start_us":79,"threads_started":1}
I20260806 06:47:44.882954 18503 consensus_queue.cc:260] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.883318 18503 ts_tablet_manager.cc:1428] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:47:44.883446 18202 heartbeater.cc:503] Master 127.6.57.126:36585 was elected leader, sending a full tablet report...
I20260806 06:47:44.883476 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:44.863842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.6.57.67:43747)
0806 06:47:44.878176 (+ 14334us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:44.878180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:44.878192 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:47:44.878240 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:44.878242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:44.878259 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:44.878363 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:47:44.878536 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.879875 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.881374 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:44.882412 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:44.883187 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:44.883275 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:44.883301 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:47:44.883343 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14291,"thread_start_us":83,"threads_started":1}
I20260806 06:47:44.884263 18416 tablet_service.cc:1467] Processing CreateTablet for tablet fc62be58bc694b228ac6048204296668 (DEFAULT_TABLE table=test-table [id=b86db676ca584486aa46fbfb7e63b4fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:47:44.884622 18416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc62be58bc694b228ac6048204296668. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:47:44.884898 18504 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: No bootstrap required, opened a new log
I20260806 06:47:44.884958 18504 ts_tablet_manager.cc:1397] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260806 06:47:44.887979 18504 raft_consensus.cc:348] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.888144 18504 raft_consensus.cc:374] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:47:44.888185 18504 raft_consensus.cc:729] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59cd1e2cc4454eb133f482c90df099, State: Initialized, Role: FOLLOWER
I20260806 06:47:44.888325 18504 consensus_queue.cc:260] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.888566 18504 ts_tablet_manager.cc:1428] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:47:44.888686 18346 heartbeater.cc:503] Master 127.6.57.126:36585 was elected leader, sending a full tablet report...
I20260806 06:47:44.889343 18509 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Bootstrap starting.
I20260806 06:47:44.890221 18509 tablet_bootstrap.cc:654] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Neither blocks nor log segments found. Creating new log.
I20260806 06:47:44.890820 18509 log.cc:824] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Log is configured to *not* fsync() on all Append() calls
I20260806 06:47:44.891618 18509 tablet_bootstrap.cc:492] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: No bootstrap required, opened a new log
I20260806 06:47:44.891717 18509 ts_tablet_manager.cc:1397] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:47:44.893379 18509 raft_consensus.cc:348] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.893532 18509 raft_consensus.cc:374] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:47:44.893594 18509 raft_consensus.cc:729] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4141a9738d4833a7855c1e7bf223ce, State: Initialized, Role: FOLLOWER
I20260806 06:47:44.893721 18509 consensus_queue.cc:260] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:44.894044 18509 ts_tablet_manager.cc:1428] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:47:44.894243 18484 heartbeater.cc:503] Master 127.6.57.126:36585 was elected leader, sending a full tablet report...
Aug 06 06:47:44 dist-test-slave-5l3k krb5kdc[17485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
W20260806 06:47:45.050267 18347 tablet.cc:2369] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:47:45.059310 18511 raft_consensus.cc:482] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:47:45.059438 18511 raft_consensus.cc:504] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:45.059849 18511 leader_election.cc:288] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc59cd1e2cc4454eb133f482c90df099 (127.6.57.66:33591), 69be8514909a42f08f736f6309b55aa3 (127.6.57.65:40497)
W20260806 06:47:45.062629 18485 tablet.cc:2369] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:47:45.064123 18502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.060132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:45045 (local address 127.6.57.65:40497)
0806 06:47:45.060173 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:45.060175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:45.060452 (+ 277us) server_negotiation.cc:407] Connection header received
0806 06:47:45.060498 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:45.060499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:45.060511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:45.060564 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:47:45.060752 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.061508 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063080 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.063681 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063726 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:45.063748 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:45.064006 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 06:47:45.064054 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:47:45.064178 18486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.060032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33591 (local address 127.6.57.67:44705)
0806 06:47:45.060091 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:45.060104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:45.060161 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:45.060412 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:45.060413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:45.060415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:47:45.060600 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:45.060602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.061600 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.061601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:45.063123 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:45.063126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.063836 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:45.063987 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:45.063990 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:47:45.064032 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 06:47:45.064066 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":285}
I20260806 06:47:45.064427 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.060171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:44705 (local address 127.6.57.66:33591)
0806 06:47:45.060236 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:45.060238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:45.060245 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:47:45.060273 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:45.060274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:45.060306 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:45.060355 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:47:45.060826 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.061507 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063210 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.063754 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063806 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:45.063814 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:45.064336 (+ 522us) server_negotiation.cc:299] Negotiation successful
0806 06:47:45.064372 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:47:45.064685 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.060053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:40497 (local address 127.6.57.67:45045)
0806 06:47:45.060364 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:45.060374 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:45.060439 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:45.060589 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:47:45.060590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:47:45.060592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:47:45.060664 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:45.060665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.061600 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.061601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:45.062965 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:47:45.062968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.063806 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.063807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:47:45.063872 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:45.063924 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 06:47:45.064563 (+ 639us) client_negotiation.cc:240] Negotiation successful
0806 06:47:45.064605 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":165,"thread_start_us":103,"threads_started":1}
I20260806 06:47:45.064651 18297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62be58bc694b228ac6048204296668" candidate_uuid: "0a4141a9738d4833a7855c1e7bf223ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59cd1e2cc4454eb133f482c90df099" is_pre_election: true
I20260806 06:47:45.064790 18297 raft_consensus.cc:2454] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4141a9738d4833a7855c1e7bf223ce in term 0.
I20260806 06:47:45.065044 18371 leader_election.cc:302] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4141a9738d4833a7855c1e7bf223ce, bc59cd1e2cc4454eb133f482c90df099; no voters:
I20260806 06:47:45.065188 18511 raft_consensus.cc:2781] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:47:45.065233 18511 raft_consensus.cc:482] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:47:45.065253 18511 raft_consensus.cc:3037] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:47:45.065196 18136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62be58bc694b228ac6048204296668" candidate_uuid: "0a4141a9738d4833a7855c1e7bf223ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69be8514909a42f08f736f6309b55aa3" is_pre_election: true
I20260806 06:47:45.065325 18136 raft_consensus.cc:2454] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4141a9738d4833a7855c1e7bf223ce in term 0.
I20260806 06:47:45.066056 18511 raft_consensus.cc:504] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:45.066195 18511 leader_election.cc:288] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [CANDIDATE]: Term 1 election: Requested vote from peers bc59cd1e2cc4454eb133f482c90df099 (127.6.57.66:33591), 69be8514909a42f08f736f6309b55aa3 (127.6.57.65:40497)
I20260806 06:47:45.066397 18136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62be58bc694b228ac6048204296668" candidate_uuid: "0a4141a9738d4833a7855c1e7bf223ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69be8514909a42f08f736f6309b55aa3"
I20260806 06:47:45.066438 18297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc62be58bc694b228ac6048204296668" candidate_uuid: "0a4141a9738d4833a7855c1e7bf223ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59cd1e2cc4454eb133f482c90df099"
I20260806 06:47:45.066476 18136 raft_consensus.cc:3037] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:47:45.066524 18297 raft_consensus.cc:3037] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:47:45.067292 18136 raft_consensus.cc:2454] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4141a9738d4833a7855c1e7bf223ce in term 1.
I20260806 06:47:45.067292 18297 raft_consensus.cc:2454] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4141a9738d4833a7855c1e7bf223ce in term 1.
I20260806 06:47:45.067464 18368 leader_election.cc:302] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4141a9738d4833a7855c1e7bf223ce, 69be8514909a42f08f736f6309b55aa3; no voters:
I20260806 06:47:45.067588 18511 raft_consensus.cc:2781] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:47:45.067781 18511 raft_consensus.cc:686] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 1 LEADER]: Becoming Leader. State: Replica: 0a4141a9738d4833a7855c1e7bf223ce, State: Running, Role: LEADER
I20260806 06:47:45.067878 18511 consensus_queue.cc:237] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } }
I20260806 06:47:45.068604 17984 catalog_manager.cc:5495] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a4141a9738d4833a7855c1e7bf223ce (127.6.57.67). New cstate: current_term: 1 leader_uuid: "0a4141a9738d4833a7855c1e7bf223ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4141a9738d4833a7855c1e7bf223ce" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 } health_report { overall_health: UNKNOWN } } }
W20260806 06:47:45.072532 18203 tablet.cc:2369] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:47:45.096729 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.094391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.6.57.67:43747)
0806 06:47:45.094454 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:47:45.094456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:47:45.094466 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:47:45.094543 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:47:45.094543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:47:45.094561 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:47:45.094645 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:47:45.094792 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.095237 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.095934 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:47:45.096134 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:47:45.096204 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:47:45.096400 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:47:45.096508 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:47:45.096609 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:47:45.096627 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 06:47:45.096646 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:47:45.098769 18136 raft_consensus.cc:1261] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 1 FOLLOWER]: Refusing update from remote peer 0a4141a9738d4833a7855c1e7bf223ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:47:45.098795 18297 raft_consensus.cc:1261] T fc62be58bc694b228ac6048204296668 P bc59cd1e2cc4454eb133f482c90df099 [term 1 FOLLOWER]: Refusing update from remote peer 0a4141a9738d4833a7855c1e7bf223ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:47:45.099236 18513 consensus_queue.cc:1035] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "69be8514909a42f08f736f6309b55aa3" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 40497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:47:45.099387 18511 consensus_queue.cc:1035] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc59cd1e2cc4454eb133f482c90df099" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:47:45.102218 18518 mvcc.cc:205] Tried to move back new op lower bound from 7315451351441428480 to 7315451351318417408. Current Snapshot: MvccSnapshot[applied={T|T < 7315451351441428480}]
I20260806 06:47:45.105219 17984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:47:45.105337 17984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:47:45.106038 17984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35c88e984c7642458d30ef6f8a091731: Sending DeleteTablet for 3 replicas of tablet fc62be58bc694b228ac6048204296668
I20260806 06:47:45.106344 18263 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62be58bc694b228ac6048204296668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:47:45 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:37778
I20260806 06:47:45.106436 18116 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62be58bc694b228ac6048204296668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:47:45 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:38566
I20260806 06:47:45.106657 18528 tablet_replica.cc:330] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3: stopping tablet replica
I20260806 06:47:45.106652 18416 tablet_service.cc:1514] Processing DeleteTablet for tablet fc62be58bc694b228ac6048204296668 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:47:45 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:34404
I20260806 06:47:45.106801 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 18066
I20260806 06:47:45.106815 18528 raft_consensus.cc:2229] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:47:45.107000 18528 raft_consensus.cc:2258] T fc62be58bc694b228ac6048204296668 P 69be8514909a42f08f736f6309b55aa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:47:45.110615 18529 tablet_replica.cc:330] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: stopping tablet replica
I20260806 06:47:45.110770 18529 raft_consensus.cc:2229] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:47:45.110929 18529 raft_consensus.cc:2258] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:47:45.111375 18529 ts_tablet_manager.cc:1905] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:47:45.113384 18529 ts_tablet_manager.cc:1918] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:47:45.113535 18529 log.cc:1196] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/fc62be58bc694b228ac6048204296668
I20260806 06:47:45.113904 18529 ts_tablet_manager.cc:1939] T fc62be58bc694b228ac6048204296668 P 0a4141a9738d4833a7855c1e7bf223ce: Deleting consensus metadata
I20260806 06:47:45.114382 17970 catalog_manager.cc:4841] TS 0a4141a9738d4833a7855c1e7bf223ce (127.6.57.67:43747): tablet fc62be58bc694b228ac6048204296668 (table test-table [id=b86db676ca584486aa46fbfb7e63b4fb]) successfully deleted
W20260806 06:47:45.114466 17967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:40497 (error 108)
W20260806 06:47:45.115062 18497 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:47:45.114834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:40497 (local address 127.0.0.1:38576)
0806 06:47:45.114881 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:45.114895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:47:45.114958 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:47:45.115006 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:40497: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:47:45.115182 17967 catalog_manager.cc:4584] TS 69be8514909a42f08f736f6309b55aa3 (127.6.57.65:40497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc62be58bc694b228ac6048204296668: Network error: Client connection negotiation failed: client connection to 127.6.57.65:40497: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260806 06:47:45.115207 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 18209
I20260806 06:47:45.123561 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 18352
I20260806 06:47:45.126839 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:47:45.126701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33591 (local address 127.0.0.1:37786)
0806 06:47:45.126749 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:47:45.126802 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.66:33591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:47:45.129122 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 17950
I20260806 06:47:45.136201 6373 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:47:45.480211 6373 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:47:45.480366 6373 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:47:45.824632 6373 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:47:45.824 UTC [17517] LOG: received smart shutdown request
2026-08-06 06:47:45.828 UTC [17517] LOG: background worker "logical replication launcher" (PID 17527) exited with exit code 1
2026-08-06 06:47:45.828 UTC [17522] LOG: shutting down
2026-08-06 06:47:45.984 UTC [17517] LOG: database system is shut down
2026-08-06T06:47:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46788 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:47:46 dist-test-slave-5l3k krb5kdc[18539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:47:46 dist-test-slave-5l3k krb5kdc[18539](info): set up 2 sockets
Aug 06 06:47:46 dist-test-slave-5l3k krb5kdc[18539](info): commencing operation
krb5kdc: starting...
W20260806 06:47:47.136076 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:47:47 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:47:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangeradmin_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangeradmin_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangerlookup_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangerlookup_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
I20260806 06:47:47.294749 6373 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXFWrbB/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/postgres -l logfile start
2026-08-06 06:47:49.682 UTC [18571] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 06:47:49.682 UTC [18571] LOG: listening on IPv4 address "127.6.57.84", port 43631
2026-08-06 06:47:49.685 UTC [18571] LOG: listening on Unix socket "/tmp/.s.PGSQL.43631"
2026-08-06 06:47:49.707 UTC [18575] LOG: database system was shut down at 2026-08-06 06:47:48 UTC
2026-08-06 06:47:49.711 UTC [18571] LOG: database system is ready to accept connections
I20260806 06:47:50.680363 6373 mini_postgres.cc:96] Postgres bound to 43631
2026-08-06 06:47:50.687 UTC [18583] FATAL: database "slave" does not exist
127.6.57.84:43631 - accepting connections
I20260806 06:47:50.687769 6373 mini_ranger.cc:161] Starting Ranger...
I20260806 06:47:50.702693 6373 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 06:47:51.164129 6373 mini_ranger.cc:87] Created ranger Postgres database
I20260806 06:47:51.164214 6373 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin
2026-08-06 06:47:51,430 [I] DB FLAVOR :POSTGRES
2026-08-06 06:47:51,430 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:47:51,430 [I] Checking connection..
2026-08-06 06:47:51,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 06:47:51,688 [I] Checking connection passed.
2026-08-06 06:47:51,688 [I] --------- Verifying version history table ---------
2026-08-06 06:47:51,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:51,946 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:47:51,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:52,207 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 06:47:52,208 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 06:47:52,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 06:47:52.449 UTC [18649] WARNING: there is no transaction in progress
2026-08-06 06:47:52,455 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 06:47:52,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 06:47:52,713 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 06:47:52,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:47:52,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 06:47:52,959 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 06:47:53,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5l3k.c.gcp-upstream.internal','N') ;"
2026-08-06 06:47:53,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:47:53,701 [I] Table x_portal_user does not exist in database ranger
2026-08-06 06:47:53,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:47:53,946 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 06:47:53,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:47:54,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:47:54,462 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 06:47:54,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 06:47:55.407 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.422 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.438 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.453 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.468 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.485 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.540 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.547 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.562 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.571 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.581 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.594 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.603 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.609 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.615 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:55.621 UTC [18761] WARNING: there is no transaction in progress
2026-08-06 06:47:56,239 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 06:47:56,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 06:47:56,479 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 06:47:56,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 06:47:56,716 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 06:47:56,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 06:47:56,974 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 06:47:56,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 06:47:57,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 06:47:57,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5l3k.c.gcp-upstream.internal';"
2026-08-06 06:47:57,699 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 06:47:57,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 06:47:57,947 [I] DB_PATCHES have already been applied
I20260806 06:47:57.953430 6373 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXFWrbB/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 06:47:57.953493 6373 mini_ranger.cc:193] Using host: 127.6.57.84
I20260806 06:47:57.957160 6373 mini_ranger.cc:227] Ranger admin URL: http://127.6.57.84:55093
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:47:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 06:47:59.087860 6373 mini_ranger.cc:160] Time spent starting Ranger: real 8.400s user 0.001s sys 0.009s
Aug 06, 2026 6:48:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:48:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 6:48:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:48:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:48:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 06:48:20.620638 6373 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.57.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.57.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangerkms_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.57.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/rangerkms_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.57.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.57.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.57.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/HTTP_127.6.57.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 06:48:20 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998900, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 06:48:20.705832 6373 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 06:48:20.715526 6373 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 06:48:21.555516 6373 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 06:48:21.555617 6373 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms
I20260806 06:48:21.555637 6373 mini_ranger_kms.cc:227] Using postgres at 127.6.57.84:43631
2026-08-06 06:48:21,721 [I] DB FLAVOR :POSTGRES
2026-08-06 06:48:21,721 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 06:48:21,721 [I] Checking connection
2026-08-06 06:48:21,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 06:48:21,985 [I] connection success
2026-08-06 06:48:21,985 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 06:48:21,985 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 06:48:21,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 06:48:22,247 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 06:48:22,248 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 06:48:22,248 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 06:48:22,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.57.84:43631/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 06:48:22,530 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 06:48:23.013437 6373 mini_ranger_kms.cc:310] Created kms service
I20260806 06:48:23.191112 6373 mini_ranger_kms.cc:326] Created kudu user
I20260806 06:48:23.233484 6373 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 06:48:23 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998900, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:23.609153 6373 mini_ranger_kms.cc:384] Added ranger policy
I20260806 06:48:23.609265 6373 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXFWrbB/build/release/bin/postgresql.jar:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXFWrbB/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 06:48:23.609287 6373 mini_ranger_kms.cc:240] Using host: 127.6.57.84
I20260806 06:48:23.613745 6373 mini_ranger_kms.cc:276] Ranger KMS PID: 18947
I20260806 06:48:23.613834 6373 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.57.84:52369
Aug 06, 2026 6:48:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:48:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:48:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:48:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:48:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:48:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 06:48:25 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998905, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:25.961735 6373 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.256s user 0.004s sys 0.145s
I20260806 06:48:25.961840 6373 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 06:48:25.961892 6373 mini_ranger_kms.cc:401] 127.6.57.84:52369/kms/v1/keys
Aug 06 06:48:26 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998905, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
Aug 06 06:48:26 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998905, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.57.84@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
Aug 06 06:48:28 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:48:28.965113 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37413
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:37413
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:29.047187 19001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:29.047384 19001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:29.047406 19001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:29.048453 19001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:29.048487 19001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:29.048502 19001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:29.048514 19001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:29.048527 19001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:29.050081 19001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.57.84:52369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:37413
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:29.050267 19001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:29.050441 19001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:29.055523 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:29.055578 19007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:29.055578 19009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:29.055609 19001 server_base.cc:1034] running on GCE node
I20260806 06:48:29.055920 19001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:29.056164 19001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:29.057340 19001 hybrid_clock.cc:648] HybridClock initialized: now 1785998909057318 us; error 41 us; skew 500 ppm
Aug 06 06:48:29 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998909, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:29.059887 19001 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:29.060357 19001 webserver.cc:460] Webserver started at http://127.6.57.126:46561/ using document root <none> and password file <none>
I20260806 06:48:29.060627 19001 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:29.060683 19001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:29.060822 19001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:48:29 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998909, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:29.104408 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "53ce1b1bab6049b38b214b94f88ad6ae"
format_stamp: "Formatted at 2026-08-06 06:48:29 on dist-test-slave-5l3k"
server_key: "b2854b65c92da8c87125e07752b1fa05"
server_key_iv: "f875768fa6eec480c6e3d0f50c662810"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:29.104820 19001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "53ce1b1bab6049b38b214b94f88ad6ae"
format_stamp: "Formatted at 2026-08-06 06:48:29 on dist-test-slave-5l3k"
server_key: "b2854b65c92da8c87125e07752b1fa05"
server_key_iv: "f875768fa6eec480c6e3d0f50c662810"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:29.106406 19001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:48:29.120371 19016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:29.120579 19001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:29.120690 19001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "53ce1b1bab6049b38b214b94f88ad6ae"
format_stamp: "Formatted at 2026-08-06 06:48:29 on dist-test-slave-5l3k"
server_key: "b2854b65c92da8c87125e07752b1fa05"
server_key_iv: "f875768fa6eec480c6e3d0f50c662810"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:29.120774 19001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:29.131916 19001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:29.133919 19001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:29.134060 19001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:29.139083 19001 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:37413
I20260806 06:48:29.139140 19071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:37413 every 8 connection(s)
I20260806 06:48:29.139479 19001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:29.140178 19072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:29.140600 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19001
I20260806 06:48:29.140709 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:48:29.140888 6373 external_mini_cluster.cc:1428] Setting key 98af614fe30782e25b0fca5d789bd02f
I20260806 06:48:29.142920 19072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Bootstrap starting.
I20260806 06:48:29.143566 19072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:29.143859 19072 log.cc:824] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:29.144559 19072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: No bootstrap required, opened a new log
I20260806 06:48:29.146579 19072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:29.146731 19072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:29.146833 19072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ce1b1bab6049b38b214b94f88ad6ae, State: Initialized, Role: FOLLOWER
I20260806 06:48:29.147001 19072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:29.147104 19072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:29.147147 19072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:29.147195 19072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 0 FOLLOWER]: Advancing to term 1
Aug 06 06:48:29 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:29.147771 19072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:29.147908 19072 leader_election.cc:302] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ce1b1bab6049b38b214b94f88ad6ae; no voters:
I20260806 06:48:29.148100 19072 leader_election.cc:288] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:48:29.148147 19077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:29.148312 19077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 LEADER]: Becoming Leader. State: Replica: 53ce1b1bab6049b38b214b94f88ad6ae, State: Running, Role: LEADER
I20260806 06:48:29.148402 19072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:29.148408 19077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:29.149120 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ce1b1bab6049b38b214b94f88ad6ae. Latest consensus state: current_term: 1 leader_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } } }
I20260806 06:48:29.149398 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:29.149142 19078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } } }
I20260806 06:48:29.149720 19078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:29.149899 19086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:29.150446 19086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:29.150815 19072 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:48:29.155512 19075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:29.141450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.6.57.126:37413)
0806 06:48:29.141643 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:29.141653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:29.141685 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:48:29.142114 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:29.142121 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:29.142167 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:29.142338 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:29.142978 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:29.144053 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:29.144872 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:29.145284 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:29.145411 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:29.147943 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:29.147947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:29.147947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:29.147959 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:29.150076 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:29.150526 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:29.150529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:29.150530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:29.150630 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:29.150853 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:29.150854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:29.150854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:29.151158 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:29.153960 (+ 2802us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:29.154239 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 06:48:29.155044 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260806 06:48:29.156111 19086 catalog_manager.cc:1347] Generated new cluster ID: 3674711500d2446f9c805efcf45e1322
I20260806 06:48:29.156184 19086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:48:29.168547 19086 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:48:29.169059 19086 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:29.172969 19086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Generated new TSK 0
I20260806 06:48:29.173143 19086 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:48:29.881716 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:29.996809 19119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:29.997056 19119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:29.997098 19119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:29.998399 19119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:29.998462 19119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:29.998529 19119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:30.000077 19119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:30.000334 19119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:30.000573 19119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:30.005496 19127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.005606 19124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.005496 19125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:30.006750 19119 server_base.cc:1034] running on GCE node
I20260806 06:48:30.007006 19119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:30.007269 19119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:30.008447 19119 hybrid_clock.cc:648] HybridClock initialized: now 1785998910008414 us; error 52 us; skew 500 ppm
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:30.011178 19119 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:30.011716 19119 webserver.cc:460] Webserver started at http://127.6.57.65:46159/ using document root <none> and password file <none>
I20260806 06:48:30.011965 19119 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:30.012046 19119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:30.012205 19119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:30.026621 19119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "7bc4b34ef12c4e75bd173752ca12cbec"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "4ea0bf1082813cf7bec590e2e7a8e6a3"
server_key_iv: "e72504f9b7254faa70f4cdf9d7c4474f"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.027197 19119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "7bc4b34ef12c4e75bd173752ca12cbec"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "4ea0bf1082813cf7bec590e2e7a8e6a3"
server_key_iv: "e72504f9b7254faa70f4cdf9d7c4474f"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.030321 19119 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 06:48:30.047088 19134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.050619 19119 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260806 06:48:30.050743 19119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "7bc4b34ef12c4e75bd173752ca12cbec"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "4ea0bf1082813cf7bec590e2e7a8e6a3"
server_key_iv: "e72504f9b7254faa70f4cdf9d7c4474f"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.050841 19119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:30.065095 19119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:30.067586 19119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:30.067726 19119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:30.068037 19119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:30.068426 19119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:30.068480 19119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.068509 19119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:30.068528 19119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.076022 19119 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39253
I20260806 06:48:30.076375 19119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:30.076970 19255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39253 every 8 connection(s)
I20260806 06:48:30.078528 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19119
I20260806 06:48:30.078612 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:48:30.078832 6373 external_mini_cluster.cc:1428] Setting key 648a953aa8ab16dd94efbac8cd82cc89
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:30.088126 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.077146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.65:54057)
0806 06:48:30.077331 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:30.077366 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:30.077937 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:30.078303 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:30.078307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:30.078616 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:30.078901 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.078908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.079924 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.079925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.081160 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.081164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.081710 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.081711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.081851 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.082408 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:30.082420 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:30.084121 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:30.086404 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.086410 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.086413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.086720 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.086973 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.086974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.086974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.087127 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.087439 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:30.087446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:30.087642 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 06:48:30.087738 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:48:30.087887 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260806 06:48:30.088855 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.077505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54057 (local address 127.6.57.126:37413)
0806 06:48:30.078085 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:30.078087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:30.078093 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:48:30.078124 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:30.078125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:30.078136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:30.078228 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:30.079119 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.079819 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.081273 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.081590 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.082082 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.084246 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:30.084247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:30.084247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:30.084255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:30.086304 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.086815 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.086816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.086816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.086877 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.087207 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.087208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.087209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.087354 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:30.088582 (+ 1228us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:30.088657 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:48:30.088743 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:48:30.090974 19256 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:30.091091 19256 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:30.091270 19256 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:30.091881 19034 ts_manager.cc:194] Registered new tserver with Master: 7bc4b34ef12c4e75bd173752ca12cbec (127.6.57.65:39253)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
I20260806 06:48:30.092790 19034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:54057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:48:30.107841 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:30.217849 19262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:30.218178 19262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:30.218263 19262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:30.219950 19262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:30.220001 19262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:30.220039 19262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:30.222071 19262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:30.222321 19262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:30.222609 19262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:30.228791 19269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.228843 19272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.228931 19270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:30.229701 19262 server_base.cc:1034] running on GCE node
I20260806 06:48:30.229905 19262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:30.230175 19262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:30.231369 19262 hybrid_clock.cc:648] HybridClock initialized: now 1785998910231332 us; error 52 us; skew 500 ppm
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:30.234577 19262 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:30.235059 19262 webserver.cc:460] Webserver started at http://127.6.57.66:37525/ using document root <none> and password file <none>
I20260806 06:48:30.235325 19262 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:30.235391 19262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:30.235558 19262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:30.269791 19262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "c67d915d47fb49e8a851b89c2e0d196b"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e0154bb2f6df2253a15f3156f4d1e885"
server_key_iv: "d27ee50d02977a71734b03142261b44c"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.270308 19262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "c67d915d47fb49e8a851b89c2e0d196b"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e0154bb2f6df2253a15f3156f4d1e885"
server_key_iv: "d27ee50d02977a71734b03142261b44c"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.272186 19262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:30.293948 19279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.294229 19262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:30.294354 19262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "c67d915d47fb49e8a851b89c2e0d196b"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e0154bb2f6df2253a15f3156f4d1e885"
server_key_iv: "d27ee50d02977a71734b03142261b44c"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.294507 19262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:30.326601 19262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:30.329144 19262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:30.329346 19262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:30.329633 19262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:30.330049 19262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:30.330138 19262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.330212 19262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:30.330271 19262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.336889 19262 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:38055
I20260806 06:48:30.337384 19262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:48:30.339741 19396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:38055 every 8 connection(s)
I20260806 06:48:30.342911 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19262
I20260806 06:48:30.343009 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:48:30.343206 6373 external_mini_cluster.cc:1428] Setting key ca3f6198dcf508798b751b7cdefbc2af
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:30.351624 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.340982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:33115 (local address 127.6.57.126:37413)
0806 06:48:30.341028 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:30.341030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:30.341345 (+ 315us) server_negotiation.cc:407] Connection header received
0806 06:48:30.342044 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:30.342045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:30.342058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:30.342090 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:30.342947 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.343715 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.345167 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.345480 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.345523 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.348228 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:30.348230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:30.348230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:30.348237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:30.349939 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.350535 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.350536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.350536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.350594 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.350887 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.350888 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.350889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.351019 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:30.351069 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:30.351421 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 06:48:30.351512 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:48:30.352306 19399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.340698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.66:33115)
0806 06:48:30.341222 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:30.341264 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:30.341912 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:30.342278 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:30.342282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:30.342557 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:30.342836 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.342843 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.343834 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.343834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.345062 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.345065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.345636 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.345637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.345775 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.346353 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:30.346363 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:30.348119 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:30.350034 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.350041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.350043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.350426 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.350675 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.350676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.350676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.350809 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.351102 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:30.351108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:30.351273 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 06:48:30.351932 (+ 659us) client_negotiation.cc:240] Negotiation successful
0806 06:48:30.352086 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":96,"threads_started":1}
I20260806 06:48:30.353094 19397 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:30.353181 19397 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:30.353364 19397 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:30.353747 19034 ts_manager.cc:194] Registered new tserver with Master: c67d915d47fb49e8a851b89c2e0d196b (127.6.57.66:38055)
I20260806 06:48:30.354202 19034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:33115
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:48:30.379488 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:30.499102 19402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:30.499369 19402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:30.499399 19402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:30.500969 19402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:30.501030 19402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:30.501078 19402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:30.503262 19402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:30.503512 19402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:30.503772 19402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:30.509255 19410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.509258 19407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.509253 19408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:30.509546 19402 server_base.cc:1034] running on GCE node
I20260806 06:48:30.509724 19402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:30.509974 19402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:30.511142 19402 hybrid_clock.cc:648] HybridClock initialized: now 1785998910511134 us; error 36 us; skew 500 ppm
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:30.513757 19402 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:30.514204 19402 webserver.cc:460] Webserver started at http://127.6.57.67:34697/ using document root <none> and password file <none>
I20260806 06:48:30.514428 19402 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:30.514514 19402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:30.514655 19402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:30.537221 19402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e3873c6145646e9becc2306c46d196ed"
server_key_iv: "0915dd3e4d38edc1390ba13f705a0119"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.537678 19402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e3873c6145646e9becc2306c46d196ed"
server_key_iv: "0915dd3e4d38edc1390ba13f705a0119"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.540814 19402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 06:48:30.558285 19417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.558720 19402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:30.558820 19402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e3873c6145646e9becc2306c46d196ed"
server_key_iv: "0915dd3e4d38edc1390ba13f705a0119"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.558899 19402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:30.580718 19402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:30.584578 19402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:30.584795 19402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:30.585078 19402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:30.585510 19402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:30.585564 19402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.585598 19402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:30.585675 19402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.594966 19402 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:33129
I20260806 06:48:30.595432 19402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:48:30.601483 19536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:33129 every 8 connection(s)
I20260806 06:48:30.603250 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19402
I20260806 06:48:30.603359 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:48:30.603557 6373 external_mini_cluster.cc:1428] Setting key c9ad164b6f4e44b1c6e81a466cfbbcc7
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:30.610127 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.596214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:35647 (local address 127.6.57.126:37413)
0806 06:48:30.596274 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:30.596277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:30.600530 (+ 4253us) server_negotiation.cc:407] Connection header received
0806 06:48:30.601295 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:30.601297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:30.601311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:30.601391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:30.602150 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.602848 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.604509 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.604860 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.604919 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.607194 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:30.607195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:30.607195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:30.607203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:30.608673 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.609089 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.609091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.609091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.609148 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:30.609406 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:30.609407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:30.609407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:30.609516 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:30.609603 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:30.609951 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 06:48:30.610035 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:48:30.610877 19539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:30.596198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.67:35647)
0806 06:48:30.600381 (+ 4183us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:30.600430 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:30.601170 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:30.601529 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:30.601534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:30.601803 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:30.602025 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.602030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.602956 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.602957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.604418 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:30.604420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:30.605012 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:30.605012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:30.605168 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:30.605741 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:30.605755 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:30.607095 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:30.608765 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.608771 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.608773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.608992 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.609217 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:30.609218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:30.609218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:30.609331 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:30.609632 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:30.609638 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:30.609789 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 06:48:30.610456 (+ 667us) client_negotiation.cc:240] Negotiation successful
0806 06:48:30.610637 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4133,"thread_start_us":82,"threads_started":1}
I20260806 06:48:30.611498 19537 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:30.611654 19537 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:30.611899 19537 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:30.612296 19034 ts_manager.cc:194] Registered new tserver with Master: 8647bf5a1b9f43c2bf0fb98ff8f99d88 (127.6.57.67:33129)
I20260806 06:48:30.612771 19034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:35647
I20260806 06:48:30.615025 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:48:30.615604 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19119
I20260806 06:48:30.626259 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19262
I20260806 06:48:30.633839 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19402
I20260806 06:48:30.639407 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19001
I20260806 06:48:30.647095 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37413
--webserver_interface=127.6.57.126
--webserver_port=46561
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:37413
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:30.728101 19540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:30.728297 19540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:30.728320 19540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:30.729408 19540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:30.729447 19540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:30.729460 19540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:30.729471 19540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:30.729482 19540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:30.731045 19540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:37413
--ranger_config_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:30.731256 19540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:30.731503 19540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:30.736280 19546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.736289 19547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.736299 19549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:30.736526 19540 server_base.cc:1034] running on GCE node
I20260806 06:48:30.736778 19540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:30.737116 19540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:30.738266 19540 hybrid_clock.cc:648] HybridClock initialized: now 1785998910738236 us; error 51 us; skew 500 ppm
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:30.740869 19540 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:30.741336 19540 webserver.cc:460] Webserver started at http://127.6.57.126:46561/ using document root <none> and password file <none>
I20260806 06:48:30.741573 19540 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:30.741652 19540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:30.775348 19540 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:30.776141 19557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.776362 19540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:30.776477 19540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "53ce1b1bab6049b38b214b94f88ad6ae"
format_stamp: "Formatted at 2026-08-06 06:48:29 on dist-test-slave-5l3k"
server_key: "b2854b65c92da8c87125e07752b1fa05"
server_key_iv: "f875768fa6eec480c6e3d0f50c662810"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.776858 19540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:30.806263 19540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:30.808182 19540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:30.808357 19540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:30.812817 19540 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:37413
I20260806 06:48:30.812884 19610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:37413 every 8 connection(s)
I20260806 06:48:30.813251 19540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:30.814735 19611 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:48:30.815448 19611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Bootstrap starting.
I20260806 06:48:30.823115 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19540
I20260806 06:48:30.823402 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:39253
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:37413
--webserver_port=46159
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:30.823868 19611 log.cc:824] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:30.826196 19611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:48:30.826577 19611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Bootstrap complete.
I20260806 06:48:30.828996 19611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:30.829258 19611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ce1b1bab6049b38b214b94f88ad6ae, State: Initialized, Role: FOLLOWER
I20260806 06:48:30.829463 19611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:30.829576 19611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:30.829635 19611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:30.829689 19611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:48:30.831017 19611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:30.831144 19611 leader_election.cc:302] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ce1b1bab6049b38b214b94f88ad6ae; no voters:
I20260806 06:48:30.831342 19611 leader_election.cc:288] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:48:30.831383 19615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:48:30.831519 19615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [term 2 LEADER]: Becoming Leader. State: Replica: 53ce1b1bab6049b38b214b94f88ad6ae, State: Running, Role: LEADER
I20260806 06:48:30.831614 19611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:30.831686 19615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } }
I20260806 06:48:30.832017 19616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } } }
I20260806 06:48:30.832036 19617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ce1b1bab6049b38b214b94f88ad6ae. Latest consensus state: current_term: 2 leader_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ce1b1bab6049b38b214b94f88ad6ae" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37413 } } }
I20260806 06:48:30.832089 19616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:30.832123 19617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:30.832587 19619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:30.833257 19619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:30.833469 19619 catalog_manager.cc:1259] Loaded cluster ID: 3674711500d2446f9c805efcf45e1322
I20260806 06:48:30.833575 19619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:48:30.833675 19611 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 06:48:30.837669 19619 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:30.838255 19619 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Loaded TSK: 0
I20260806 06:48:30.838589 19619 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:48:30.932555 19614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:30.932883 19614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:30.932979 19614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:30.934269 19614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:30.934337 19614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:30.934414 19614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:30.936080 19614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:39253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=46159
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:30.936414 19614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:30.936690 19614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:30.944814 19649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.945322 19650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:30.944867 19652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:30.945971 19614 server_base.cc:1034] running on GCE node
I20260806 06:48:30.946188 19614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:30.946416 19614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:30.947577 19614 hybrid_clock.cc:648] HybridClock initialized: now 1785998910947560 us; error 58 us; skew 500 ppm
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:30.950266 19614 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:30.950806 19614 webserver.cc:460] Webserver started at http://127.6.57.65:46159/ using document root <none> and password file <none>
I20260806 06:48:30.951053 19614 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:30.951141 19614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:48:30 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:30.968638 19614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:30.969894 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.970118 19614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:48:30.970227 19614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "7bc4b34ef12c4e75bd173752ca12cbec"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "4ea0bf1082813cf7bec590e2e7a8e6a3"
server_key_iv: "e72504f9b7254faa70f4cdf9d7c4474f"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:30.970674 19614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:30.985014 19614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:30.987489 19614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:30.987709 19614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:30.987982 19614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:30.988427 19614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:30.988550 19614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.988669 19614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:30.988765 19614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:30.996826 19614 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39253
I20260806 06:48:30.997282 19614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:30.998976 19778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39253 every 8 connection(s)
I20260806 06:48:31.006652 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19614
I20260806 06:48:31.006997 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:38055
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:37413
--webserver_port=37525
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998910, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:31.013975 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.000829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:46951 (local address 127.6.57.126:37413)
0806 06:48:31.001107 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:31.001117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:31.001157 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:48:31.001808 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:31.001834 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:31.001891 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:31.002046 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:31.003638 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.004978 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.006161 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.006706 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.006828 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.009658 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:31.009662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:31.009662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:31.009676 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:31.011758 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.012316 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.012319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.012320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.012433 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.012775 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.012776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.012777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.013110 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:31.013216 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:31.013639 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 06:48:31.013765 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
I20260806 06:48:31.014365 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.000739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.65:46951)
0806 06:48:31.000920 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:31.000971 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:31.001493 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:31.002135 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:31.002138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:31.002317 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:31.002569 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.002581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.005097 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.005098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.006073 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.006076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.006954 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.006955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.007110 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.007729 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:31.007740 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:31.009524 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:31.011870 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.011878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.011881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.012211 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.012527 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.012528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.012528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.012685 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.013207 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:31.013215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:31.013392 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 06:48:31.013546 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 06:48:31.013736 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260806 06:48:31.015499 19780 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:31.015658 19780 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.015919 19780 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.017175 19780 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:37413 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:48:31.017338 19780 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.017490 19780 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:31.018559 19780 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.018679 19780 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.019133 19780 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:48:31.117233 19784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:31.117493 19784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:31.117527 19784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:31.119199 19784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:31.119294 19784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:31.119352 19784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:31.121480 19784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:38055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=37525
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:31.121837 19784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:31.122094 19784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:31.127526 19792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:31.127619 19790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:31.127883 19789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:31.128152 19784 server_base.cc:1034] running on GCE node
I20260806 06:48:31.128367 19784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:31.128616 19784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:31.129773 19784 hybrid_clock.cc:648] HybridClock initialized: now 1785998911129748 us; error 49 us; skew 500 ppm
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:31.132740 19784 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:31.133291 19784 webserver.cc:460] Webserver started at http://127.6.57.66:37525/ using document root <none> and password file <none>
I20260806 06:48:31.133549 19784 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:31.133630 19784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:31.155619 19784 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 06:48:31.156714 19800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.156872 19784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:31.156955 19784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "c67d915d47fb49e8a851b89c2e0d196b"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e0154bb2f6df2253a15f3156f4d1e885"
server_key_iv: "d27ee50d02977a71734b03142261b44c"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:31.157424 19784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:31.173388 19784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:31.175848 19784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:31.175998 19784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:31.176244 19784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:31.176654 19784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:31.176750 19784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.176796 19784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:31.176829 19784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.184288 19784 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:38055
I20260806 06:48:31.184405 19917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:38055 every 8 connection(s)
I20260806 06:48:31.184738 19784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:31.194348 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19784
I20260806 06:48:31.194682 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.57.84:52369/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:33129
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:37413
--webserver_port=34697
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:31.195137 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.185730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:39665 (local address 127.6.57.126:37413)
0806 06:48:31.185819 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:31.185821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:31.185888 (+ 67us) server_negotiation.cc:407] Connection header received
0806 06:48:31.186585 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:31.186586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:31.186597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:31.186641 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:31.187402 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.188156 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.189635 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.189822 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.189876 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.191991 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:31.191993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:31.191993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:31.192001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:31.193601 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.194109 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.194110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.194110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.194176 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.194439 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.194440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.194440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.194597 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:31.194680 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:31.194962 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 06:48:31.195049 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:48:31.195289 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.185449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.66:39665)
0806 06:48:31.185643 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:31.185804 (+ 161us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:31.186440 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:31.186738 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:31.186743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:31.186959 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:31.187268 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.187275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.188299 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.188300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.189528 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.189532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.189934 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.189936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.190061 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.190514 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:31.190523 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:31.191892 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:31.193746 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.193753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.193755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.194042 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.194249 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.194250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.194250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.194365 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.194683 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:31.194690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:31.194830 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 06:48:31.194930 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:48:31.195063 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260806 06:48:31.197484 19918 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:31.197588 19918 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.197805 19918 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.198623 19918 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:37413 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:48:31.198717 19918 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.198791 19918 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:31.199167 19918 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.199251 19918 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.199589 19918 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:48:31.302639 19921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:31.302898 19921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:31.302932 19921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:31.304488 19921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:31.304545 19921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:31.304584 19921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:31.306648 19921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.57.84:52369/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:33129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=34697
--tserver_master_addrs=127.6.57.126:37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:31.306864 19921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:31.307165 19921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:31.313320 19930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:31.313627 19927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:31.313858 19926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:31.313915 19921 server_base.cc:1034] running on GCE node
I20260806 06:48:31.314167 19921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:31.314441 19921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:31.315656 19921 hybrid_clock.cc:648] HybridClock initialized: now 1785998911315637 us; error 32 us; skew 500 ppm
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:31.319337 19921 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:31.319893 19921 webserver.cc:460] Webserver started at http://127.6.57.67:34697/ using document root <none> and password file <none>
I20260806 06:48:31.320183 19921 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:31.320319 19921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:31.339924 19921 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:31.340991 19940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.341210 19921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:31.341393 19921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88"
format_stamp: "Formatted at 2026-08-06 06:48:30 on dist-test-slave-5l3k"
server_key: "e3873c6145646e9becc2306c46d196ed"
server_key_iv: "0915dd3e4d38edc1390ba13f705a0119"
server_key_version: "kuduclusterkey@0"
I20260806 06:48:31.344954 19921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:31.354094 19921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:31.356356 19921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:31.356503 19921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:31.356742 19921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:31.357084 19921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:31.357136 19921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.357158 19921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:31.357172 19921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:31.364393 19921 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:33129
I20260806 06:48:31.364491 20062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:33129 every 8 connection(s)
I20260806 06:48:31.364812 19921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:31.372380 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 19921
I20260806 06:48:31.374966 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.365588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:41639 (local address 127.6.57.126:37413)
0806 06:48:31.365674 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:31.365676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:31.365910 (+ 234us) server_negotiation.cc:407] Connection header received
0806 06:48:31.366612 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:31.366614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:31.366624 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:31.366676 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:31.367345 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.367816 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.369199 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.369368 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.369413 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.371611 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:31.371613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:31.371613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:31.371622 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:31.373501 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.373937 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.373938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.373939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.373997 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.374262 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.374263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.374263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.374377 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:31.374451 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:31.374797 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 06:48:31.374875 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:48:31.375133 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.365566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:37413 (local address 127.6.57.67:41639)
0806 06:48:31.365771 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:31.365818 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:31.366485 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:31.366750 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:31.366754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:31.366981 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:31.367258 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.367264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.367907 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.367908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.369114 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:31.369117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.369454 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.369455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:31.369578 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.370009 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:31.370018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:31.371500 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:31.373571 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.373576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.373578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.373871 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.374076 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:31.374077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:31.374077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:31.374187 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:31.374443 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:31.374449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:31.374640 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 06:48:31.374753 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:48:31.374893 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260806 06:48:31.375731 20063 heartbeater.cc:347] Connected to a master server at 127.6.57.126:37413
I20260806 06:48:31.375849 20063 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.376040 20063 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.376765 20063 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:37413 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 06:48:31.376894 20063 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.376976 20063 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:31.377326 20063 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:31.377444 20063 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
W20260806 06:48:31.377831 20063 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:48:31.387089 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:31.372789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.6.57.126:37413)
0806 06:48:31.372896 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:31.372898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:31.373707 (+ 809us) server_negotiation.cc:407] Connection header received
0806 06:48:31.373774 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:31.373775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:31.373788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:31.373868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:31.374612 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.375412 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.376596 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:31.376824 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:31.376875 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:31.378381 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:31.378382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:31.378382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:31.378390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:31.380101 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.382541 (+ 2440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.382542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.382543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.382601 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:31.383447 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:31.383448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:31.383448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:31.383561 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:31.383749 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:31.386763 (+ 3014us) server_negotiation.cc:299] Negotiation successful
0806 06:48:31.386857 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
Aug 06 06:48:31 dist-test-slave-5l3k krb5kdc[18539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:32.019320 19780 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:32.019490 19780 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:32.019936 19575 ts_manager.cc:194] Registered new tserver with Master: 7bc4b34ef12c4e75bd173752ca12cbec (127.6.57.65:39253)
I20260806 06:48:32.020648 19575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:46951
I20260806 06:48:32.199772 19918 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:32.199925 19918 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:32.200330 19575 ts_manager.cc:194] Registered new tserver with Master: c67d915d47fb49e8a851b89c2e0d196b (127.6.57.66:38055)
I20260806 06:48:32.200778 19575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:39665
Aug 06 06:48:32 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
I20260806 06:48:32.378067 20063 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:32.378224 20063 heartbeater.cc:511] Master 127.6.57.126:37413 requested a full tablet report, sending...
I20260806 06:48:32.378551 19575 ts_manager.cc:194] Registered new tserver with Master: 8647bf5a1b9f43c2bf0fb98ff8f99d88 (127.6.57.67:33129)
I20260806 06:48:32.379072 19575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:41639
I20260806 06:48:32.381407 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:48:32.390187 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.382384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55940 (local address 127.6.57.126:37413)
0806 06:48:32.382540 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.382543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.382613 (+ 70us) server_negotiation.cc:407] Connection header received
0806 06:48:32.382682 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.382683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.382696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.382770 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:32.383416 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.384172 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.385132 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.385436 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.385503 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.387054 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:32.387055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:32.387056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:32.387063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:32.388899 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:32.389305 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:32.389306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:32.389306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:32.389372 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:32.389600 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:32.389601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:32.389601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:32.389733 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:32.389831 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.389997 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.390083 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260806 06:48:32.392274 19575 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:48:32.394423 19575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:48:32.403661 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.397543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:38055 (local address 127.0.0.1:35244)
0806 06:48:32.398591 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.398601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:32.398659 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:32.399201 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:32.399205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:32.399206 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:32.399265 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.399267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.400833 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.400834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.402137 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.402139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.403379 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.403380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.403455 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.403466 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:48:32.403537 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:48:32.403584 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
I20260806 06:48:32.403877 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.398063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.6.57.66:38055)
0806 06:48:32.398933 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.398938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.398947 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:32.398979 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.398981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.399013 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.399079 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:32.399339 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.400739 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.402238 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.403229 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.403294 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.403346 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.403765 (+ 419us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.403801 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":55,"threads_started":1}
I20260806 06:48:32.404582 19849 tablet_service.cc:1467] Processing CreateTablet for tablet b4d109b89138483090bccb3d1a06892e (DEFAULT_TABLE table=test-table [id=50b4aad8e91b4ab1a12f9c192a49790a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:32.404919 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d109b89138483090bccb3d1a06892e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:32.407735 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.397823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39253 (local address 127.0.0.1:40114)
0806 06:48:32.398769 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.398777 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:32.398818 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:32.399421 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:32.399423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:32.399425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:32.399482 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.399483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.401034 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.401036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.402344 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.402348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.407496 (+ 5148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.407497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.407560 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.407564 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:32.407624 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:48:32.407663 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":91,"threads_started":1}
I20260806 06:48:32.408118 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.398124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.6.57.65:39253)
0806 06:48:32.398371 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.398377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.399134 (+ 757us) server_negotiation.cc:407] Connection header received
0806 06:48:32.399192 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.399195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.399213 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.399296 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:32.399573 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.400924 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.402450 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.403262 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.403335 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.403393 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.407988 (+ 4595us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.408037 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":63,"threads_started":1}
I20260806 06:48:32.409096 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.397544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:33129 (local address 127.0.0.1:51546)
0806 06:48:32.398454 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.398544 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:32.398661 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:32.403431 (+ 4770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:32.403432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:32.403435 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:32.403476 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.403477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.405445 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.405446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.406542 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.406545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.408869 (+ 2324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.408870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.408933 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.408937 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:32.408984 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:48:32.409022 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":92,"threads_started":1}
I20260806 06:48:32.409268 19711 tablet_service.cc:1467] Processing CreateTablet for tablet b4d109b89138483090bccb3d1a06892e (DEFAULT_TABLE table=test-table [id=50b4aad8e91b4ab1a12f9c192a49790a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:32.409451 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.397904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.6.57.67:33129)
0806 06:48:32.398305 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.398311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.398800 (+ 489us) server_negotiation.cc:407] Connection header received
0806 06:48:32.398880 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.398883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.398904 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.399037 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:32.403562 (+ 4525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.405336 (+ 1774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.406637 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.407340 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.407411 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.407461 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.409328 (+ 1867us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.409374 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":74,"threads_started":1}
I20260806 06:48:32.409611 19711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d109b89138483090bccb3d1a06892e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:32.409946 20096 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Bootstrap starting.
I20260806 06:48:32.410784 20096 tablet_bootstrap.cc:654] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:32.411193 20096 log.cc:824] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:32.411537 19990 tablet_service.cc:1467] Processing CreateTablet for tablet b4d109b89138483090bccb3d1a06892e (DEFAULT_TABLE table=test-table [id=50b4aad8e91b4ab1a12f9c192a49790a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:32.411844 19990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d109b89138483090bccb3d1a06892e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:32.411978 20096 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: No bootstrap required, opened a new log
I20260806 06:48:32.412034 20096 ts_tablet_manager.cc:1397] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:32.413877 20098 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Bootstrap starting.
I20260806 06:48:32.414554 20098 tablet_bootstrap.cc:654] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:32.414844 20098 log.cc:824] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:32.415041 20096 raft_consensus.cc:348] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.415205 20096 raft_consensus.cc:374] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:32.415239 20096 raft_consensus.cc:729] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c67d915d47fb49e8a851b89c2e0d196b, State: Initialized, Role: FOLLOWER
I20260806 06:48:32.415657 20100 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Bootstrap starting.
I20260806 06:48:32.415491 20096 consensus_queue.cc:260] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.416113 20096 ts_tablet_manager.cc:1428] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 06:48:32.416266 19918 heartbeater.cc:503] Master 127.6.57.126:37413 was elected leader, sending a full tablet report...
I20260806 06:48:32.416605 20100 tablet_bootstrap.cc:654] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:32.416898 20100 log.cc:824] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:32.417600 20100 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: No bootstrap required, opened a new log
I20260806 06:48:32.417663 20100 ts_tablet_manager.cc:1397] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:32.420333 20100 raft_consensus.cc:348] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.420480 20100 raft_consensus.cc:374] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:32.420509 20100 raft_consensus.cc:729] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8647bf5a1b9f43c2bf0fb98ff8f99d88, State: Initialized, Role: FOLLOWER
I20260806 06:48:32.420627 20100 consensus_queue.cc:260] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.420904 20063 heartbeater.cc:503] Master 127.6.57.126:37413 was elected leader, sending a full tablet report...
I20260806 06:48:32.421470 20098 tablet_bootstrap.cc:492] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: No bootstrap required, opened a new log
I20260806 06:48:32.421561 20098 ts_tablet_manager.cc:1397] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260806 06:48:32.420854 20100 ts_tablet_manager.cc:1428] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:48:32.423946 20098 raft_consensus.cc:348] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.424095 20098 raft_consensus.cc:374] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:32.424125 20098 raft_consensus.cc:729] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc4b34ef12c4e75bd173752ca12cbec, State: Initialized, Role: FOLLOWER
I20260806 06:48:32.424320 20098 consensus_queue.cc:260] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.424585 20098 ts_tablet_manager.cc:1428] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:48:32.424742 19780 heartbeater.cc:503] Master 127.6.57.126:37413 was elected leader, sending a full tablet report...
W20260806 06:48:32.436019 19919 tablet.cc:2369] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:48:32.500161 19781 tablet.cc:2369] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 06:48:32 dist-test-slave-5l3k krb5kdc[18539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998911, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for HTTP/127.6.57.84@KRBTEST.COM
W20260806 06:48:32.618441 20064 tablet.cc:2369] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:48:32.638435 20104 raft_consensus.cc:482] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:32.638630 20104 raft_consensus.cc:504] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.639109 20104 leader_election.cc:288] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c67d915d47fb49e8a851b89c2e0d196b (127.6.57.66:38055), 8647bf5a1b9f43c2bf0fb98ff8f99d88 (127.6.57.67:33129)
I20260806 06:48:32.654517 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.639745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:38633 (local address 127.6.57.66:38055)
0806 06:48:32.639795 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.639797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.650710 (+ 10913us) server_negotiation.cc:407] Connection header received
0806 06:48:32.650966 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.650968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.650982 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.651013 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:32.651296 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.652021 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.653539 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.654096 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.654146 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.654158 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.654375 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.654417 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:48:32.655014 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.639250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:38055 (local address 127.6.57.65:38633)
0806 06:48:32.650613 (+ 11363us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.650631 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:32.650891 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:32.651115 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:32.651116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:32.651119 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:32.651206 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.651208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.652144 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.652145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.653441 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.653443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.654220 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.654221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.654291 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.654295 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:32.654872 (+ 577us) client_negotiation.cc:240] Negotiation successful
0806 06:48:32.654915 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11319,"thread_start_us":105,"threads_started":1}
I20260806 06:48:32.655560 19871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d109b89138483090bccb3d1a06892e" candidate_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c67d915d47fb49e8a851b89c2e0d196b" is_pre_election: true
I20260806 06:48:32.655743 19871 raft_consensus.cc:2454] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc4b34ef12c4e75bd173752ca12cbec in term 0.
I20260806 06:48:32.655987 19664 leader_election.cc:302] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc4b34ef12c4e75bd173752ca12cbec, c67d915d47fb49e8a851b89c2e0d196b; no voters:
I20260806 06:48:32.656141 20104 raft_consensus.cc:2781] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:48:32.656186 20104 raft_consensus.cc:482] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:48:32.656209 20104 raft_consensus.cc:3037] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:32.657254 20104 raft_consensus.cc:504] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.657401 20104 leader_election.cc:288] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [CANDIDATE]: Term 1 election: Requested vote from peers c67d915d47fb49e8a851b89c2e0d196b (127.6.57.66:38055), 8647bf5a1b9f43c2bf0fb98ff8f99d88 (127.6.57.67:33129)
I20260806 06:48:32.657720 19871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d109b89138483090bccb3d1a06892e" candidate_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c67d915d47fb49e8a851b89c2e0d196b"
I20260806 06:48:32.657822 19871 raft_consensus.cc:3037] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:32.658823 19871 raft_consensus.cc:2454] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc4b34ef12c4e75bd173752ca12cbec in term 1.
I20260806 06:48:32.659072 19664 leader_election.cc:302] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bc4b34ef12c4e75bd173752ca12cbec, c67d915d47fb49e8a851b89c2e0d196b; no voters:
I20260806 06:48:32.659108 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.639617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:57625 (local address 127.6.57.67:33129)
0806 06:48:32.639668 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.639671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.650819 (+ 11148us) server_negotiation.cc:407] Connection header received
0806 06:48:32.650890 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.650891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.650908 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.650981 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:32.655961 (+ 4980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.656623 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.658025 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.658619 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.658668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.658679 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.658984 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.659029 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 06:48:32.659320 20104 raft_consensus.cc:2781] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:32.659356 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.639516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:33129 (local address 127.6.57.65:57625)
0806 06:48:32.650758 (+ 11242us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.650766 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:32.650827 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:32.651041 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:32.651043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:32.651045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:32.651162 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.651164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.656730 (+ 5566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.656731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.657917 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:32.657920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.658739 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.658739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:32.658882 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.658886 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:32.659184 (+ 298us) client_negotiation.cc:240] Negotiation successful
0806 06:48:32.659226 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11229,"thread_start_us":87,"threads_started":1}
I20260806 06:48:32.659464 20104 raft_consensus.cc:686] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [term 1 LEADER]: Becoming Leader. State: Replica: 7bc4b34ef12c4e75bd173752ca12cbec, State: Running, Role: LEADER
I20260806 06:48:32.659556 20104 consensus_queue.cc:237] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } }
I20260806 06:48:32.660377 19575 catalog_manager.cc:5495] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bc4b34ef12c4e75bd173752ca12cbec (127.6.57.65). New cstate: current_term: 1 leader_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39253 } health_report { overall_health: HEALTHY } } }
I20260806 06:48:32.661377 20011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d109b89138483090bccb3d1a06892e" candidate_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" is_pre_election: true
I20260806 06:48:32.661525 20011 raft_consensus.cc:2454] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bc4b34ef12c4e75bd173752ca12cbec in term 0.
I20260806 06:48:32.661667 20010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d109b89138483090bccb3d1a06892e" candidate_uuid: "7bc4b34ef12c4e75bd173752ca12cbec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88"
I20260806 06:48:32.661725 20010 raft_consensus.cc:3037] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:32.662739 20010 raft_consensus.cc:2454] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bc4b34ef12c4e75bd173752ca12cbec in term 1.
I20260806 06:48:32.696002 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.692602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.6.57.65:39253)
0806 06:48:32.692663 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:32.692665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:32.692673 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:48:32.692715 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:32.692716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:32.692738 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:32.692807 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:48:32.692997 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.693736 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.694853 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:32.695162 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:32.695229 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:32.695440 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:48:32.695574 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:48:32.695662 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:32.695903 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 06:48:32.695924 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:48:32.698398 20010 raft_consensus.cc:1261] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 1 FOLLOWER]: Refusing update from remote peer 7bc4b34ef12c4e75bd173752ca12cbec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:32.698885 20109 consensus_queue.cc:1035] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [LEADER]: Connected to new peer: Peer: permanent_uuid: "8647bf5a1b9f43c2bf0fb98ff8f99d88" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:32.701303 19871 raft_consensus.cc:1261] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 1 FOLLOWER]: Refusing update from remote peer 7bc4b34ef12c4e75bd173752ca12cbec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:32.701659 20109 consensus_queue.cc:1035] T b4d109b89138483090bccb3d1a06892e P 7bc4b34ef12c4e75bd173752ca12cbec [LEADER]: Connected to new peer: Peer: permanent_uuid: "c67d915d47fb49e8a851b89c2e0d196b" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 38055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:32.705806 20112 mvcc.cc:205] Tried to move back new op lower bound from 7315451546409881600 to 7315451546254082048. Current Snapshot: MvccSnapshot[applied={T|T < 7315451546409881600}]
I20260806 06:48:32.707041 20113 mvcc.cc:205] Tried to move back new op lower bound from 7315451546409881600 to 7315451546254082048. Current Snapshot: MvccSnapshot[applied={T|T < 7315451546409881600}]
I20260806 06:48:32.708288 20116 mvcc.cc:205] Tried to move back new op lower bound from 7315451546409881600 to 7315451546254082048. Current Snapshot: MvccSnapshot[applied={T|T < 7315451546409881600}]
I20260806 06:48:32.712826 19575 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:32.712924 19575 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:32.713762 19575 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53ce1b1bab6049b38b214b94f88ad6ae: Sending DeleteTablet for 3 replicas of tablet b4d109b89138483090bccb3d1a06892e
I20260806 06:48:32.714246 19849 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d109b89138483090bccb3d1a06892e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:32 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:35244
I20260806 06:48:32.714504 19711 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d109b89138483090bccb3d1a06892e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:32 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:40114
I20260806 06:48:32.714890 19990 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d109b89138483090bccb3d1a06892e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:32 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:51546
I20260806 06:48:32.715538 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19614
I20260806 06:48:32.718618 20126 tablet_replica.cc:330] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: stopping tablet replica
I20260806 06:48:32.718660 20123 tablet_replica.cc:330] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: stopping tablet replica
I20260806 06:48:32.718747 20126 raft_consensus.cc:2229] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:32.718747 20123 raft_consensus.cc:2229] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:32.718852 20126 raft_consensus.cc:2258] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:32.718852 20123 raft_consensus.cc:2258] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:32.719465 20123 ts_tablet_manager.cc:1905] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:32.727053 20126 ts_tablet_manager.cc:1905] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:32.728657 20126 ts_tablet_manager.cc:1918] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:32.728813 20126 log.cc:1196] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/b4d109b89138483090bccb3d1a06892e
I20260806 06:48:32.729161 20126 ts_tablet_manager.cc:1939] T b4d109b89138483090bccb3d1a06892e P 8647bf5a1b9f43c2bf0fb98ff8f99d88: Deleting consensus metadata
I20260806 06:48:32.729656 19559 catalog_manager.cc:4841] TS 8647bf5a1b9f43c2bf0fb98ff8f99d88 (127.6.57.67:33129): tablet b4d109b89138483090bccb3d1a06892e (table test-table [id=50b4aad8e91b4ab1a12f9c192a49790a]) successfully deleted
I20260806 06:48:32.731571 20123 ts_tablet_manager.cc:1918] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:32.731789 20123 log.cc:1196] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/b4d109b89138483090bccb3d1a06892e
I20260806 06:48:32.732337 20123 ts_tablet_manager.cc:1939] T b4d109b89138483090bccb3d1a06892e P c67d915d47fb49e8a851b89c2e0d196b: Deleting consensus metadata
I20260806 06:48:32.733038 19560 catalog_manager.cc:4841] TS c67d915d47fb49e8a851b89c2e0d196b (127.6.57.66:38055): tablet b4d109b89138483090bccb3d1a06892e (table test-table [id=50b4aad8e91b4ab1a12f9c192a49790a]) successfully deleted
W20260806 06:48:32.733681 19561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:39253 (error 108)
I20260806 06:48:32.734982 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19784
I20260806 06:48:32.746695 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19921
I20260806 06:48:32.758963 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:32.758809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39253 (local address 127.0.0.1:40118)
0806 06:48:32.758875 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:32.758916 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 06:48:32.759261 19561 catalog_manager.cc:4584] TS 7bc4b34ef12c4e75bd173752ca12cbec (127.6.57.65:39253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4d109b89138483090bccb3d1a06892e: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39253: connect: Connection refused (error 111)
I20260806 06:48:32.759765 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 19540
I20260806 06:48:32.766026 6373 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 06:48:33.110354 6373 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 06:48:33.110466 6373 mini_ranger.cc:66] Stopping Ranger...
I20260806 06:48:33.454577 6373 mini_ranger.cc:68] Stopped Ranger
2026-08-06 06:48:33.454 UTC [18571] LOG: received smart shutdown request
2026-08-06 06:48:33.458 UTC [18571] LOG: background worker "logical replication launcher" (PID 18581) exited with exit code 1
2026-08-06 06:48:33.459 UTC [18576] LOG: shutting down
2026-08-06 06:48:33.621 UTC [18571] LOG: database system is shut down
2026-08-06T06:48:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47645 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:48:33 dist-test-slave-5l3k krb5kdc[20137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:48:33 dist-test-slave-5l3k krb5kdc[20137](info): set up 2 sockets
Aug 06 06:48:33 dist-test-slave-5l3k krb5kdc[20137](info): commencing operation
krb5kdc: starting...
W20260806 06:48:34.797765 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:48:34 dist-test-slave-5l3k krb5kdc[20137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:48:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:48:34.921708 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46637
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:35.001979 20150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:35.002194 20150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:35.002240 20150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:35.003414 20150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:35.003458 20150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:35.003471 20150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:35.003506 20150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:35.003531 20150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:35.004961 20150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:35.005187 20150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:35.005396 20150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:35.010210 20158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:35.010219 20156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:35.010468 20150 server_base.cc:1034] running on GCE node
W20260806 06:48:35.010241 20155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:35.010893 20150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:35.011260 20150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:35.012432 20150 hybrid_clock.cc:648] HybridClock initialized: now 1785998915012422 us; error 28 us; skew 500 ppm
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:35.015002 20150 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:35.015625 20150 webserver.cc:460] Webserver started at http://127.6.57.126:41827/ using document root <none> and password file <none>
I20260806 06:48:35.015874 20150 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:35.015944 20150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:35.016080 20150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:35.017156 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "e91c3d47e24c4d0182cf0c9236b10bbb"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "0236b1d431f27902d37d226c061a2ba8"
server_key_iv: "f012ef188be8c4e67706bc9a18e50f0e"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.017524 20150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "e91c3d47e24c4d0182cf0c9236b10bbb"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "0236b1d431f27902d37d226c061a2ba8"
server_key_iv: "f012ef188be8c4e67706bc9a18e50f0e"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.018993 20150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:48:35.019795 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:35.019999 20150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:35.020100 20150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "e91c3d47e24c4d0182cf0c9236b10bbb"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "0236b1d431f27902d37d226c061a2ba8"
server_key_iv: "f012ef188be8c4e67706bc9a18e50f0e"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.020195 20150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:35.037482 20150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:35.039840 20150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:35.040007 20150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:35.043958 20150 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46637
I20260806 06:48:35.043995 20219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46637 every 8 connection(s)
I20260806 06:48:35.044353 20150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:35.045222 20220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:35.046203 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20150
I20260806 06:48:35.046278 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:48:35.046461 6373 external_mini_cluster.cc:1428] Setting key 281c9bfe1bd85328f95708462c300182
I20260806 06:48:35.046868 20220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.125@KRBTEST.COM; defaulting to no policy
I20260806 06:48:35.057807 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.047635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.6.57.126:46637)
0806 06:48:35.048278 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.048281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.048310 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:48:35.048597 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.048599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.048624 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.048705 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.049445 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.050752 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.051680 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.052231 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.052354 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.054439 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.054447 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.054447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.054455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.056132 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.056576 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.056579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.056579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.056661 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.056885 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.056886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.056886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.057083 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.057172 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.057353 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.057540 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":87,"threads_started":1}
I20260806 06:48:35.057807 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.047618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46637 (local address 127.0.0.1:47960)
0806 06:48:35.047791 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.047832 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.048450 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.048758 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.048761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.049114 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.049341 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.049348 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.050852 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.050853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.051587 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.051589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.052321 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.052321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.052376 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.052870 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.052878 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.054353 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.056208 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.056215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.056217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.056472 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.056731 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.056732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.056733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.056819 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.057164 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.057170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.057229 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.057329 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.057590 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":111,"thread_start_us":99,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.125@KRBTEST.COM" created.
I20260806 06:48:35.058705 20220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.059082 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.058952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59286)
0806 06:48:35.058997 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.059057 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:48:35.059202 20170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
I20260806 06:48:35.059576 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.059491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59296)
0806 06:48:35.059534 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.059557 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:48:35.059675 20220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:33177: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
I20260806 06:48:35.072799 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:33177
--webserver_interface=127.6.57.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:35.108923 20220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } attempt: 1
I20260806 06:48:35.109611 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.109460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59308)
0806 06:48:35.109529 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.109577 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:48:35.109997 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.109904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59312)
0806 06:48:35.109950 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.109979 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:48:35.110134 20220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:33177: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
W20260806 06:48:35.152441 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:35.152632 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:35.152652 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:35.153890 20227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:35.153931 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:35.153946 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:35.153981 20227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:35.154008 20227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:35.155548 20227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:33177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.57.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:35.155817 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:35.156055 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:35.160871 20235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:35.160880 20233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:35.160878 20232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:35.161456 20227 server_base.cc:1034] running on GCE node
I20260806 06:48:35.161743 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:35.162017 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:35.163187 20227 hybrid_clock.cc:648] HybridClock initialized: now 1785998915163166 us; error 36 us; skew 500 ppm
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:35.165506 20227 init.cc:377] Logged in from keytab as kudu/127.6.57.125@KRBTEST.COM (short username kudu)
I20260806 06:48:35.165961 20227 webserver.cc:460] Webserver started at http://127.6.57.125:40953/ using document root <none> and password file <none>
I20260806 06:48:35.166188 20227 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:35.166234 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:35.166381 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:35.167439 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data/instance:
uuid: "92f72aee7b2b47f2a260c226710be560"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "fb98abf18b4a6edd725e93790ef4c88f"
server_key_iv: "5e963e806969d11f737a92bb8f87d6fa"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.167805 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal/instance:
uuid: "92f72aee7b2b47f2a260c226710be560"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "fb98abf18b4a6edd725e93790ef4c88f"
server_key_iv: "5e963e806969d11f737a92bb8f87d6fa"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.169134 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:48:35.169819 20242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:35.169983 20227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:35.170078 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "92f72aee7b2b47f2a260c226710be560"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "fb98abf18b4a6edd725e93790ef4c88f"
server_key_iv: "5e963e806969d11f737a92bb8f87d6fa"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.170171 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:35.178313 20220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } attempt: 2
I20260806 06:48:35.178428 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:35.178707 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.178582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59322)
0806 06:48:35.178644 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.178681 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:48:35.179116 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.179000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59330)
0806 06:48:35.179060 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.179087 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 06:48:35.179206 20220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:33177: Network error: Client connection negotiation failed: client connection to 127.6.57.125:33177: connect: Connection refused (error 111)
I20260806 06:48:35.180444 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:35.180589 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:35.184355 20227 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.125:33177
I20260806 06:48:35.184383 20297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.125:33177 every 8 connection(s)
I20260806 06:48:35.184700 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
I20260806 06:48:35.185596 20298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:35.187139 20298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.187341 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20227
I20260806 06:48:35.187446 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-1/wal/instance
I20260806 06:48:35.187671 6373 external_mini_cluster.cc:1428] Setting key d1b281dba16044f75874b95324dee2a5
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.124@KRBTEST.COM; defaulting to no policy
I20260806 06:48:35.196329 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.188146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.6.57.126:46637)
0806 06:48:35.188212 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.188215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.188224 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:35.188617 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.188619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.188631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.188714 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.189232 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.189811 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.191057 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.191384 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.191459 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.193435 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.193437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.193438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.193446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.195013 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.195395 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.195398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.195398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.195460 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.195699 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.195701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.195701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.195839 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.195912 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.196136 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.196224 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:48:35.196506 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.187849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46637 (local address 127.0.0.1:47972)
0806 06:48:35.188022 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.188057 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.188519 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.188760 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.188763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.188978 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.189150 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.189154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.189904 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.189906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.190970 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.190974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.191473 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.191474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.191570 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.192045 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.192053 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.193345 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.195099 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.195106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.195108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.195315 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.195533 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.195533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.195533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.195629 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.195913 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.195918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.196019 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.196104 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.196237 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260806 06:48:35.197126 20298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.124@KRBTEST.COM" created.
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:48:35.204895 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.197399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59342)
0806 06:48:35.197454 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.197467 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.197509 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.197796 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.197797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.197938 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.197981 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.197982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.199141 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.199142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.199602 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.199604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.200069 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.200070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.200150 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.200534 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.200539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.201943 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.203744 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.203746 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.203746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.203983 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.204190 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.204191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.204192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.204256 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.204509 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.204510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.204559 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.204645 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.204744 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:48:35.204905 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.197496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.6.57.125:33177)
0806 06:48:35.197650 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.197654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.197669 (+ 15us) server_negotiation.cc:407] Connection header received
0806 06:48:35.197685 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.197687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.197698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.197796 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.198060 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.199046 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.199696 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.199982 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.200069 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.202040 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.202048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.202049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.202055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.203637 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.204067 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.204069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.204069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.204121 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.204320 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.204321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.204321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.204440 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.204498 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.204667 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.204789 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260806 06:48:35.205518 20298 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.205905 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.205747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:38992)
0806 06:48:35.205801 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.205867 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:48:35.206051 20243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
I20260806 06:48:35.206552 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.206396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39000)
0806 06:48:35.206456 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.206525 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 06:48:35.206728 20298 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:43241: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
I20260806 06:48:35.215195 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:43241
--webserver_interface=127.6.57.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:35.255996 20298 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } attempt: 1
I20260806 06:48:35.256601 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.256411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39004)
0806 06:48:35.256498 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.256558 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 06:48:35.257020 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.256907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39010)
0806 06:48:35.256959 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.256995 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:48:35.257206 20298 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:43241: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
I20260806 06:48:35.270401 20220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } attempt: 3
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:48:35.278323 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.270901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.6.57.125:33177)
0806 06:48:35.270974 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.270976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.270986 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:48:35.271015 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.271016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.271031 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.271089 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.271442 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.271933 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.272760 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.272975 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.273070 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.275614 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.275615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.275616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.275622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.277110 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.277511 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.277512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.277512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.277574 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.277786 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.277787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.277787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.277903 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.277978 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.278145 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.278224 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 06:48:35.278317 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.270815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59344)
0806 06:48:35.270864 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.270875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.270942 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.271108 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.271109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.271280 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.271346 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.271348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.272035 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.272036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.272671 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.272674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.273090 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.273091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.273172 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.273657 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.273663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.275494 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.277240 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.277241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.277241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.277419 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.277643 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.277644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.277644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.277704 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.277982 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.277983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.278028 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.278106 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.278210 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 06:48:35.278709 20220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.279121 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.278952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39012)
0806 06:48:35.279022 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.279069 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:48:35.279484 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.279381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39014)
0806 06:48:35.279417 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.279450 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 06:48:35.279600 20220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:43241: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
W20260806 06:48:35.304632 20305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:35.304857 20305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:35.304883 20305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:35.306110 20305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:35.306155 20305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:35.306171 20305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:35.306185 20305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:35.306247 20305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:35.307786 20305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46637,127.6.57.125:33177,127.6.57.124:43241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:43241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.57.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:35.308079 20305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:35.308327 20305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:48:35.310738 20220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } attempt: 1
I20260806 06:48:35.311221 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.311084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39018)
0806 06:48:35.311155 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.311194 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 06:48:35.311571 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.311470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39022)
0806 06:48:35.311523 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.311548 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:48:35.311659 20220 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:43241: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
W20260806 06:48:35.313479 20313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:35.313475 20310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:35.313503 20311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:35.313731 20305 server_base.cc:1034] running on GCE node
I20260806 06:48:35.314010 20305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:35.314311 20305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:35.315474 20305 hybrid_clock.cc:648] HybridClock initialized: now 1785998915315451 us; error 37 us; skew 500 ppm
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:35.317936 20305 init.cc:377] Logged in from keytab as kudu/127.6.57.124@KRBTEST.COM (short username kudu)
I20260806 06:48:35.318373 20305 webserver.cc:460] Webserver started at http://127.6.57.124:36883/ using document root <none> and password file <none>
I20260806 06:48:35.318651 20305 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:35.318722 20305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:35.318878 20305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:35.320026 20305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data/instance:
uuid: "469f1edbec6c4a0a963be41128b3b0d1"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "e7bea48aaf6106a1f25b676cbb225860"
server_key_iv: "7917de4220ee5126c03eb796c021fc69"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.320395 20305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal/instance:
uuid: "469f1edbec6c4a0a963be41128b3b0d1"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "e7bea48aaf6106a1f25b676cbb225860"
server_key_iv: "7917de4220ee5126c03eb796c021fc69"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.321694 20305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:48:35.322388 20320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:35.322569 20305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:35.322677 20305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "469f1edbec6c4a0a963be41128b3b0d1"
format_stamp: "Formatted at 2026-08-06 06:48:35 on dist-test-slave-5l3k"
server_key: "e7bea48aaf6106a1f25b676cbb225860"
server_key_iv: "7917de4220ee5126c03eb796c021fc69"
server_key_version: "encryptionkey@0"
I20260806 06:48:35.322781 20305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:35.325438 20298 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } attempt: 2
I20260806 06:48:35.325841 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.325708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39026)
0806 06:48:35.325779 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.325807 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:48:35.326323 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.326231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39040)
0806 06:48:35.326273 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.326298 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 06:48:35.326434 20298 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:43241: Network error: Client connection negotiation failed: client connection to 127.6.57.124:43241: connect: Connection refused (error 111)
I20260806 06:48:35.338426 20305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:35.340795 20305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:35.340906 20305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:35.344890 20305 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.124:43241
I20260806 06:48:35.344960 20374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.124:43241 every 8 connection(s)
I20260806 06:48:35.345304 20305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
I20260806 06:48:35.346226 20375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:35.347838 20375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.350248 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20305
I20260806 06:48:35.350384 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785998763187424-6373-0/minicluster-data/master-2/wal/instance
I20260806 06:48:35.350617 6373 external_mini_cluster.cc:1428] Setting key cd948ea0854b2c8bd8714d469108724a
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:48:35.358201 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.348565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.6.57.126:46637)
0806 06:48:35.348629 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.348631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.348831 (+ 200us) server_negotiation.cc:407] Connection header received
0806 06:48:35.349280 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.349280 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.349290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.349347 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.349948 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.350575 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.352117 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.352428 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.352518 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.355141 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.355142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.355143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.355150 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.356592 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.357096 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.357097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.357097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.357173 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.357471 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.357472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.357473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.357619 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.357684 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.358015 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.358101 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 06:48:35.358337 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.348519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46637 (local address 127.0.0.1:47974)
0806 06:48:35.348708 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.348749 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.349201 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.349422 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.349426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.349680 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.349864 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.349868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.350682 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.350683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.351990 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.351994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.352530 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.352531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.352641 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.353108 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.353118 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.355022 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.356679 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.356686 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.356689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.356981 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.357257 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.357258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.357259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.357387 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.357702 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.357709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.357880 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.357964 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.358082 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":103,"threads_started":1}
I20260806 06:48:35.359073 20375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.360599 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.351012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.6.57.124:43241)
0806 06:48:35.351218 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.351222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.351264 (+ 42us) server_negotiation.cc:407] Connection header received
0806 06:48:35.351336 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.351338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.351362 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.351424 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.352115 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.353119 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.354098 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.354567 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.354640 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.357314 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.357321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.357321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.357328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.359449 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.359893 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.359896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.359897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.359960 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.360158 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.360158 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.360159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.360273 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.360383 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.360482 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.360536 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":120,"threads_started":1}
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:35.366417 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.359346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:33177 (local address 127.0.0.1:59358)
0806 06:48:35.359494 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.359505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.359539 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.359671 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.359672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.359783 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.359826 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.359827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.360761 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.360762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.361500 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.361503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.362001 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.362002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.362077 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.362427 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.362432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.363730 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.365411 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.365412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.365412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.365563 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.365768 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.365769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.365769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.365835 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.366097 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.366098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.366159 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.366234 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.366314 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260806 06:48:35.366465 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.359444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.6.57.125:33177)
0806 06:48:35.359531 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.359533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.359543 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:48:35.359590 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.359591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.359602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.359671 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.359952 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.360660 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.361575 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.361918 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.361997 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.363804 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.363805 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.363806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.363811 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.365303 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.365635 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.365636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.365636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.365695 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.365897 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.365899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.365899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.366028 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.366104 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.366288 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.366350 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:48:35.366793 20375 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:48:35.369434 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.361399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.6.57.126:46637)
0806 06:48:35.361436 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.361437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.361453 (+ 16us) server_negotiation.cc:407] Connection header received
0806 06:48:35.361473 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.361474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.361482 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.361552 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.362162 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.362865 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.363719 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.363946 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.364022 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.366150 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.366151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.366151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.366158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.367748 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.368120 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.368121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.368122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.368182 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.368376 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.368377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.368377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.368500 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.369225 (+ 725us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.369267 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.369344 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:48:35.373834 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.367063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39060)
0806 06:48:35.367091 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.367100 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.367138 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.367252 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.367252 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.367375 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.367395 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.367396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.368120 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.368121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.368854 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.368857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.369188 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.369189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.369238 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.369756 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.369761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.370948 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.372711 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.372712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.372713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.372955 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.373187 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.373188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.373188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.373258 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.373495 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.373496 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.373538 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.373637 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.373746 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260806 06:48:35.374187 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.367092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.6.57.124:43241)
0806 06:48:35.367131 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.367133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.367141 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:48:35.367192 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.367193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.367201 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.367240 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.367480 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.368040 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.368933 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.369118 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.369162 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.371054 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.371055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.371055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.371061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.372616 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.373067 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.373068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.373068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.373117 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.373327 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.373328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.373328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.373427 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.373478 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.373659 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.373822 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":25}
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:48:35.375816 20375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: Bootstrap starting.
I20260806 06:48:35.376881 20375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:35.377238 20375 log.cc:824] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:35.377878 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.370060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.6.57.125:33177)
0806 06:48:35.370112 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.370114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.370121 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:48:35.370165 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.370166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.370175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.370216 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.370712 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.371250 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.371869 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.372059 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.372170 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.374869 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.374870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.374871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.374877 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.376482 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.376854 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.376855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.376856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.376912 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.377230 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.377231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.377231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.377357 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.377532 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.377741 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.377821 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:48:35.378099 20375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: No bootstrap required, opened a new log
I20260806 06:48:35.379834 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:48:35.380019 20375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.380152 20375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:35.380196 20375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469f1edbec6c4a0a963be41128b3b0d1, State: Initialized, Role: FOLLOWER
I20260806 06:48:35.380312 20375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.380574 20381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.380623 20375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:35.380668 20381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:48:35.382929 20400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:48:35.382969 20400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:48:35.382862 20305 master_runner.cc:186] Error getting master registration for 127.6.57.126:46637: OK, instance_id { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" instance_seqno: 1785998915042879 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:48:35.385376 20305 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:48:35.386782 20220 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } attempt: 2
I20260806 06:48:35.388986 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.381491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.6.57.124:43241)
0806 06:48:35.381529 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.381530 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.382297 (+ 767us) server_negotiation.cc:407] Connection header received
0806 06:48:35.382393 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.382394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.382404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.382432 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.383009 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.383644 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.384454 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.384675 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.384750 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.386041 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.386043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.386043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.386050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.387666 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.388067 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.388068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.388069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.388125 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.388342 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.388343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.388343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.388512 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.388620 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.388809 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.388887 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:48:35.389937 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.381599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.6.57.126:46637)
0806 06:48:35.381637 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.381638 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.381645 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:48:35.381665 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.381666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.381675 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.381731 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.382498 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.383185 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.384161 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.384444 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.384514 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.386202 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.386204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.386204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.386211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.387874 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.388494 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.388495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.388496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.388555 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.388769 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.388770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.388770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.388903 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.388986 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.389769 (+ 783us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.389854 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:48:35.392967 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.381300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59378 (local address 127.6.57.125:33177)
0806 06:48:35.381339 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.381341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.381633 (+ 292us) server_negotiation.cc:407] Connection header received
0806 06:48:35.381731 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.381732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.381741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.381770 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.385848 (+ 4078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.386594 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.387620 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.387909 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.387974 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.390155 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.390156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.390156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.390162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.391725 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.392117 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.392118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.392119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.392168 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.392378 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.392379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.392379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.392493 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.392596 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.392792 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.392866 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:48:35.395990 20222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.387832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39076)
0806 06:48:35.387919 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.387928 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.387961 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.389398 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.389399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.389535 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.389565 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.389566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.390416 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.390417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.391168 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.391172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.391611 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.391612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.391677 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.392056 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.392061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.393546 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.395174 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.395175 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.395175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.395335 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.395522 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.395522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.395522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.395567 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.395791 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.395791 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.395820 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.395879 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.395937 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 06:48:35.396268 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.388031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.6.57.124:43241)
0806 06:48:35.389118 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.389120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.389126 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:48:35.389144 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.389145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.389155 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.389199 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.389652 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.390317 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.391242 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.391521 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.391598 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.393627 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.393628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.393629 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.393635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.395091 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.395403 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.395404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.395404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.395459 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.395621 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.395622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.395622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.395733 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.395794 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.395905 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.395972 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":291,"threads_started":1}
I20260806 06:48:35.397542 20220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: Bootstrap starting.
I20260806 06:48:35.398138 20220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:35.398449 20220 log.cc:824] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:35.399287 20220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: No bootstrap required, opened a new log
I20260806 06:48:35.400686 20220 raft_consensus.cc:348] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.400786 20220 raft_consensus.cc:374] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:35.400808 20220 raft_consensus.cc:729] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91c3d47e24c4d0182cf0c9236b10bbb, State: Initialized, Role: FOLLOWER
I20260806 06:48:35.400933 20220 consensus_queue.cc:260] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.401182 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.401247 20220 sys_catalog.cc:564] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:35.401259 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:48:35.403025 20418 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:48:35.403081 20418 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:48:35.403743 20150 master_runner.cc:186] Error getting master registration for 127.6.57.125:33177: OK, instance_id { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" instance_seqno: 1785998915183328 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:48:35.406584 20150 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:48:35.417611 20298 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } attempt: 3
Aug 06 06:48:35 dist-test-slave-5l3k krb5kdc[20137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998915, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:48:35.425654 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.417899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:43241 (local address 127.0.0.1:39084)
0806 06:48:35.417948 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:35.417958 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:35.417994 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:35.418177 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:35.418178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:35.418356 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:35.418407 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.418409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.419364 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.419365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.420141 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:35.420143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.420628 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.420629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:35.420686 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.421162 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:35.421167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:35.422689 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:35.424548 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.424550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.424551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.424766 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.424974 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:35.424974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:35.424975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:35.425050 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:35.425347 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:35.425348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:35.425393 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 06:48:35.425465 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:48:35.425555 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:48:35.425671 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:35.417973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.6.57.124:43241)
0806 06:48:35.418038 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:35.418040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:35.418048 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:48:35.418069 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:35.418070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:35.418092 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:35.418155 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:35.418508 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.419267 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.420232 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:35.420528 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:35.420601 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:35.422775 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:35.422776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:35.422776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:35.422784 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:35.424459 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.424846 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.424847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.424848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.424908 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:35.425125 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:35.425125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:35.425126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:35.425247 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:35.425313 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:35.425504 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 06:48:35.425575 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 06:48:35.427667 20298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: Bootstrap starting.
I20260806 06:48:35.428391 20298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:35.428699 20298 log.cc:824] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:35.429555 20298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: No bootstrap required, opened a new log
I20260806 06:48:35.431628 20298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.431751 20298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:35.431782 20298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f72aee7b2b47f2a260c226710be560, State: Initialized, Role: FOLLOWER
I20260806 06:48:35.431885 20298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.432190 20298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:35.432148 20421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.432332 20421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:48:35.433995 20432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:48:35.434111 20432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:48:35.435993 20227 master_runner.cc:263] No leader master found from master 92f72aee7b2b47f2a260c226710be560
I20260806 06:48:35.436048 20227 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 06:48:35.540741 20172 tablet.cc:2369] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:48:35.541093 20405 raft_consensus.cc:482] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:35.541188 20405 raft_consensus.cc:504] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.541440 20405 leader_election.cc:288] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92f72aee7b2b47f2a260c226710be560 (127.6.57.125:33177), 469f1edbec6c4a0a963be41128b3b0d1 (127.6.57.124:43241)
I20260806 06:48:35.541703 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f72aee7b2b47f2a260c226710be560" is_pre_election: true
I20260806 06:48:35.541700 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469f1edbec6c4a0a963be41128b3b0d1" is_pre_election: true
I20260806 06:48:35.541842 20348 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91c3d47e24c4d0182cf0c9236b10bbb in term 0.
I20260806 06:48:35.541841 20271 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e91c3d47e24c4d0182cf0c9236b10bbb in term 0.
I20260806 06:48:35.542016 20167 leader_election.cc:302] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469f1edbec6c4a0a963be41128b3b0d1, e91c3d47e24c4d0182cf0c9236b10bbb; no voters:
I20260806 06:48:35.542135 20405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:48:35.542169 20405 raft_consensus.cc:482] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:48:35.542188 20405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:35.542958 20405 raft_consensus.cc:504] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.543116 20405 leader_election.cc:288] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [CANDIDATE]: Term 1 election: Requested vote from peers 92f72aee7b2b47f2a260c226710be560 (127.6.57.125:33177), 469f1edbec6c4a0a963be41128b3b0d1 (127.6.57.124:43241)
I20260806 06:48:35.543290 20348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469f1edbec6c4a0a963be41128b3b0d1"
I20260806 06:48:35.543363 20271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f72aee7b2b47f2a260c226710be560"
I20260806 06:48:35.543376 20348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:35.543445 20271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:35.544257 20348 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91c3d47e24c4d0182cf0c9236b10bbb in term 1.
I20260806 06:48:35.544260 20271 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e91c3d47e24c4d0182cf0c9236b10bbb in term 1.
I20260806 06:48:35.544414 20167 leader_election.cc:302] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469f1edbec6c4a0a963be41128b3b0d1, e91c3d47e24c4d0182cf0c9236b10bbb; no voters:
I20260806 06:48:35.544482 20405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:35.544628 20405 raft_consensus.cc:686] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [term 1 LEADER]: Becoming Leader. State: Replica: e91c3d47e24c4d0182cf0c9236b10bbb, State: Running, Role: LEADER
I20260806 06:48:35.544713 20405 consensus_queue.cc:237] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } }
I20260806 06:48:35.545050 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91c3d47e24c4d0182cf0c9236b10bbb. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.545193 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:35.545418 20436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:35.545601 20436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:35.546906 20271 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [term 1 FOLLOWER]: Refusing update from remote peer e91c3d47e24c4d0182cf0c9236b10bbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:35.546916 20348 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [term 1 FOLLOWER]: Refusing update from remote peer e91c3d47e24c4d0182cf0c9236b10bbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:35.547204 20434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:35.548182 20434 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:35.548471 20421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91c3d47e24c4d0182cf0c9236b10bbb. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.548579 20421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:48:35.549432 20381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91c3d47e24c4d0182cf0c9236b10bbb. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.549549 20381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:48:35.550593 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.550680 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:35.550752 20381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.550820 20381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:48:35.551214 20440 mvcc.cc:205] Tried to move back new op lower bound from 7315451558078164992 to 7315451558071697408. Current Snapshot: MvccSnapshot[applied={T|T < 7315451558078164992}]
I20260806 06:48:35.551875 20434 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.551959 20434 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:35.552385 20421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91c3d47e24c4d0182cf0c9236b10bbb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46637 } } peers { permanent_uuid: "92f72aee7b2b47f2a260c226710be560" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 33177 } } peers { permanent_uuid: "469f1edbec6c4a0a963be41128b3b0d1" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 43241 } } }
I20260806 06:48:35.552493 20421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:48:35.552624 20436 catalog_manager.cc:1347] Generated new cluster ID: 4174393c7d6246aea97d23d6356218b0
I20260806 06:48:35.552697 20436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:48:35.570659 20436 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:48:35.571273 20436 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:35.578971 20436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e91c3d47e24c4d0182cf0c9236b10bbb: Generated new TSK 0
I20260806 06:48:35.579145 20436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:48:36.383385 20400 catalog_manager.cc:1259] Loaded cluster ID: 4174393c7d6246aea97d23d6356218b0
I20260806 06:48:36.383460 20400 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: loading cluster ID for follower catalog manager: success
I20260806 06:48:36.384243 20400 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: acquiring CA information for follower catalog manager: success
I20260806 06:48:36.384491 20400 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 469f1edbec6c4a0a963be41128b3b0d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:48:36.434634 20432 catalog_manager.cc:1259] Loaded cluster ID: 4174393c7d6246aea97d23d6356218b0
I20260806 06:48:36.434702 20432 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: loading cluster ID for follower catalog manager: success
I20260806 06:48:36.435477 20432 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: acquiring CA information for follower catalog manager: success
I20260806 06:48:36.435766 20432 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92f72aee7b2b47f2a260c226710be560: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:48:36.630157 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20150
I20260806 06:48:36.637087 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20227
I20260806 06:48:36.643193 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20305
2026-08-06T06:48:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2908 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:48:36 dist-test-slave-5l3k krb5kdc[20451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:48:36 dist-test-slave-5l3k krb5kdc[20451](info): set up 2 sockets
Aug 06 06:48:36 dist-test-slave-5l3k krb5kdc[20451](info): commencing operation
krb5kdc: starting...
W20260806 06:48:37.697439 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:48:37 dist-test-slave-5l3k krb5kdc[20451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:48:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:48:37.829144 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37431
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:37431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:37.912782 20465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:37.913028 20465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:37.913053 20465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:37.914206 20465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:37.914256 20465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:37.914273 20465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:37.914286 20465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:37.914300 20465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:37.915719 20465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:37431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:37.915918 20465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:37.916129 20465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:37.921654 20470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:37.921696 20471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:37.921655 20473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:37.921934 20465 server_base.cc:1034] running on GCE node
I20260806 06:48:37.922211 20465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:37.922595 20465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:37.923768 20465 hybrid_clock.cc:648] HybridClock initialized: now 1785998917923743 us; error 37 us; skew 500 ppm
Aug 06 06:48:37 dist-test-slave-5l3k krb5kdc[20451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998917, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:37.926430 20465 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:37.926930 20465 webserver.cc:460] Webserver started at http://127.6.57.126:41419/ using document root <none> and password file <none>
I20260806 06:48:37.927184 20465 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:37.927261 20465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:37.927425 20465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:37.928588 20465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "2375033ce38945e6a1981eac7d695771"
format_stamp: "Formatted at 2026-08-06 06:48:37 on dist-test-slave-5l3k"
server_key: "cdecefe8a1f4471dd6eead97aad4c3bf"
server_key_iv: "5d50f030fa91ed21da99215002948984"
server_key_version: "encryptionkey@0"
I20260806 06:48:37.928992 20465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "2375033ce38945e6a1981eac7d695771"
format_stamp: "Formatted at 2026-08-06 06:48:37 on dist-test-slave-5l3k"
server_key: "cdecefe8a1f4471dd6eead97aad4c3bf"
server_key_iv: "5d50f030fa91ed21da99215002948984"
server_key_version: "encryptionkey@0"
I20260806 06:48:37.930428 20465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 06:48:37.931268 20480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:37.931519 20465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:37.931665 20465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "2375033ce38945e6a1981eac7d695771"
format_stamp: "Formatted at 2026-08-06 06:48:37 on dist-test-slave-5l3k"
server_key: "cdecefe8a1f4471dd6eead97aad4c3bf"
server_key_iv: "5d50f030fa91ed21da99215002948984"
server_key_version: "encryptionkey@0"
I20260806 06:48:37.931751 20465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:37.972460 20465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:37.974545 20465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:37.974706 20465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:37.978843 20465 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:37431
I20260806 06:48:37.978917 20536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:37431 every 8 connection(s)
I20260806 06:48:37.979297 20465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:37.980264 20537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:37.982604 20537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771: Bootstrap starting.
I20260806 06:48:37.983150 20537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:37.983393 20537 log.cc:824] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:37.984066 20537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771: No bootstrap required, opened a new log
I20260806 06:48:37.984395 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20465
I20260806 06:48:37.984501 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:48:37.984720 6373 external_mini_cluster.cc:1428] Setting key e7c6c5c28bde6d37fcc487bd80fee995
I20260806 06:48:37.986522 20537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } }
I20260806 06:48:37.986709 20537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:37.986750 20537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2375033ce38945e6a1981eac7d695771, State: Initialized, Role: FOLLOWER
I20260806 06:48:37.986897 20537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } }
I20260806 06:48:37.986990 20537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:37.987059 20537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:37.987141 20537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:37.987691 20537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } }
I20260806 06:48:37.987797 20537 leader_election.cc:302] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2375033ce38945e6a1981eac7d695771; no voters:
I20260806 06:48:37.988018 20537 leader_election.cc:288] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:48:37.988051 20542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:37.988164 20542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [term 1 LEADER]: Becoming Leader. State: Replica: 2375033ce38945e6a1981eac7d695771, State: Running, Role: LEADER
I20260806 06:48:37.988250 20542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } }
I20260806 06:48:37.988289 20537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:37.988569 20542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2375033ce38945e6a1981eac7d695771. Latest consensus state: current_term: 1 leader_uuid: "2375033ce38945e6a1981eac7d695771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } } }
I20260806 06:48:37.988659 20542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:37.988580 20543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2375033ce38945e6a1981eac7d695771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2375033ce38945e6a1981eac7d695771" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 37431 } } }
I20260806 06:48:37.988821 20543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771 [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:37.989603 20547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:37.990072 20547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:37.991741 20547 catalog_manager.cc:1347] Generated new cluster ID: bdc8c22c9b824b4ea005d9bb32df262c
I20260806 06:48:37.991797 20547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:48:37 dist-test-slave-5l3k krb5kdc[20451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:37.996734 20541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:37.985261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.6.57.126:37431)
0806 06:48:37.985477 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:37.985484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:37.985510 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:48:37.985989 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:37.985995 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:37.986039 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:37.986157 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:37.987015 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:37.988647 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:37.989606 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:37.990250 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:37.990353 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:37.993022 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:37.993025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:37.993026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:37.993038 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:37.995124 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:37.995515 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:37.995518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:37.995519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:37.995627 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:37.995824 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:37.995825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:37.995826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:37.996156 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:37.996308 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:37.996404 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:48:37.996535 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260806 06:48:37.998347 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:48:37.998400 20547 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:48:37.998919 20547 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:38.007426 20547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2375033ce38945e6a1981eac7d695771: Generated new TSK 0
I20260806 06:48:38.007591 20547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:48:38.052395 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20465
2026-08-06T06:48:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1409 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:48:38 dist-test-slave-5l3k krb5kdc[20566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:48:38 dist-test-slave-5l3k krb5kdc[20566](info): set up 2 sockets
Aug 06 06:48:38 dist-test-slave-5l3k krb5kdc[20566](info): commencing operation
krb5kdc: starting...
W20260806 06:48:39.188438 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:48:39.307232 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:39.387662 20579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:39.387842 20579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:39.387868 20579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:39.389005 20579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:39.389057 20579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:39.389072 20579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:39.389122 20579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:39.389154 20579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:39.390657 20579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:39.390890 20579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:39.391108 20579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:39.395736 20587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.395745 20584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.395748 20585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.396003 20579 server_base.cc:1034] running on GCE node
I20260806 06:48:39.396265 20579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:39.396553 20579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:39.397707 20579 hybrid_clock.cc:648] HybridClock initialized: now 1785998919397694 us; error 28 us; skew 500 ppm
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:39.405830 20579 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:39.406314 20579 webserver.cc:460] Webserver started at http://127.6.57.126:44745/ using document root <none> and password file <none>
I20260806 06:48:39.406628 20579 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:39.406697 20579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:39.406850 20579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:39.407846 20579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "53ca7ee193b042c7b50d854acd2bb32a"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "3c618e11e3dd60395b84ea8311701a15"
server_key_iv: "2ad7219851d4d96919e3dd710e998373"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.408211 20579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "53ca7ee193b042c7b50d854acd2bb32a"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "3c618e11e3dd60395b84ea8311701a15"
server_key_iv: "2ad7219851d4d96919e3dd710e998373"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.409504 20579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:48:39.410239 20594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.410455 20579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:39.410593 20579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "53ca7ee193b042c7b50d854acd2bb32a"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "3c618e11e3dd60395b84ea8311701a15"
server_key_iv: "2ad7219851d4d96919e3dd710e998373"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.410684 20579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:39.427213 20579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:39.429024 20579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:39.429193 20579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:39.433480 20579 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46793
I20260806 06:48:39.433512 20647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46793 every 8 connection(s)
I20260806 06:48:39.433840 20579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:39.434891 20648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:39.437273 20648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap starting.
I20260806 06:48:39.437887 20648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:39.438181 20648 log.cc:824] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:39.438915 20648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: No bootstrap required, opened a new log
I20260806 06:48:39.440474 20648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:39.440616 20648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:39.440641 20648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Initialized, Role: FOLLOWER
I20260806 06:48:39.440817 20648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:39.440881 20648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:39.440922 20648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:39.440971 20648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:39.441769 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20579
I20260806 06:48:39.441885 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:48:39.441922 20648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:39.442034 20648 leader_election.cc:302] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ca7ee193b042c7b50d854acd2bb32a; no voters:
I20260806 06:48:39.442121 6373 external_mini_cluster.cc:1428] Setting key 164ba43bc9f74a1371aec0a93b5a303f
I20260806 06:48:39.442198 20648 leader_election.cc:288] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:48:39.442369 20652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:39.442462 20648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:39.442550 20652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 LEADER]: Becoming Leader. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Running, Role: LEADER
I20260806 06:48:39.442744 20652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:39.443214 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:39.443296 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:39.443418 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ca7ee193b042c7b50d854acd2bb32a. Latest consensus state: current_term: 1 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:39.443460 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:39.443890 20663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:39.444416 20663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:39.445839 20663 catalog_manager.cc:1347] Generated new cluster ID: b14d4a08646c40bfb12a5a497ed48741
I20260806 06:48:39.445887 20663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:39.453051 20663 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:48:39.453624 20663 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:39.454674 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.442653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.6.57.126:46793)
0806 06:48:39.442959 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.442970 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.443154 (+ 184us) server_negotiation.cc:407] Connection header received
0806 06:48:39.443373 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.443382 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.443442 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.443560 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:39.444529 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.445885 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.446899 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.447560 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.447677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.450435 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:39.450439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:39.450440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:39.450451 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:39.453065 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.453451 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.453454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.453455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.453554 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.453778 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.453779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.453779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.454022 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:39.454174 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.454286 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.454419 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260806 06:48:39.458330 20663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Generated new TSK 0
I20260806 06:48:39.458509 20663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:48:39.486718 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:39.567848 20676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:39.568086 20676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:39.568122 20676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:39.569242 20676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:39.569300 20676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:39.569344 20676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:39.570833 20676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:39.571112 20676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:39.571341 20676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:39.576088 20682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.576086 20681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.576094 20684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.576376 20676 server_base.cc:1034] running on GCE node
I20260806 06:48:39.576633 20676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:39.576870 20676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:39.578064 20676 hybrid_clock.cc:648] HybridClock initialized: now 1785998919578048 us; error 26 us; skew 500 ppm
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:39.580597 20676 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:39.581029 20676 webserver.cc:460] Webserver started at http://127.6.57.65:36169/ using document root <none> and password file <none>
I20260806 06:48:39.581324 20676 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:39.581396 20676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:39.581570 20676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:39.582679 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "a40736f190974f67a50611d283db3a90"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "0260ba0769b4436659d178ecd4e326f9"
server_key_iv: "56cf1e9b0b7febff14f33128f67f16bf"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.583045 20676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "a40736f190974f67a50611d283db3a90"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "0260ba0769b4436659d178ecd4e326f9"
server_key_iv: "56cf1e9b0b7febff14f33128f67f16bf"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.584396 20676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:48:39.585101 20691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.585271 20676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:39.585363 20676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "a40736f190974f67a50611d283db3a90"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "0260ba0769b4436659d178ecd4e326f9"
server_key_iv: "56cf1e9b0b7febff14f33128f67f16bf"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.585443 20676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:39.599354 20676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:39.601056 20676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:39.601207 20676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:39.601459 20676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:39.601771 20676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:39.601824 20676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.601856 20676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:39.601883 20676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.607612 20676 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:43781
I20260806 06:48:39.607656 20809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:43781 every 8 connection(s)
I20260806 06:48:39.607961 20676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:39.611725 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20676
I20260806 06:48:39.611822 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:48:39.612010 6373 external_mini_cluster.cc:1428] Setting key 284a902d439e694c73fb52c6fec90cd3
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:39.617713 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.608673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:44633 (local address 127.6.57.126:46793)
0806 06:48:39.608734 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.608736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.608911 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:48:39.609370 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.609370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.609380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.609433 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:39.610430 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.610979 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.612138 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.612404 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.612482 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.614779 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:39.614780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:39.614781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:39.614787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:39.616199 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.616712 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.616713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.616713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.616763 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.617093 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.617095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.617095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.617207 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:39.617271 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.617586 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.617653 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 06:48:39.618043 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.608629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:44633)
0806 06:48:39.608786 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.608827 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.609290 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.609501 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.609504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.610174 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:39.610353 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.610357 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.611061 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.611062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.612053 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.612058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.612491 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.612493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.612620 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.613196 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:39.613204 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:39.614688 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:39.616281 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.616287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.616290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.616635 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.616853 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.616854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.616855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.617003 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.617283 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:39.617291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:39.617449 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.617566 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.617750 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 06:48:39.618623 20810 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:39.618712 20810 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:39.618892 20810 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:39.619413 20611 ts_manager.cc:194] Registered new tserver with Master: a40736f190974f67a50611d283db3a90 (127.6.57.65:43781)
I20260806 06:48:39.620066 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:44633
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:48:39.637297 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:39.718591 20817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:39.718814 20817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:39.718850 20817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:39.719947 20817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:39.720009 20817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:39.720052 20817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:39.721549 20817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:39.721818 20817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:39.722047 20817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:39.727065 20825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.727090 20823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.727120 20817 server_base.cc:1034] running on GCE node
W20260806 06:48:39.727106 20822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.727419 20817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:39.727613 20817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:39.728828 20817 hybrid_clock.cc:648] HybridClock initialized: now 1785998919728828 us; error 40 us; skew 500 ppm
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:39.731328 20817 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:39.731803 20817 webserver.cc:460] Webserver started at http://127.6.57.66:40435/ using document root <none> and password file <none>
I20260806 06:48:39.732002 20817 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:39.732043 20817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:39.732139 20817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:39.733075 20817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "395b0bed56f647f896b5f22d43baba6d"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "7c9070e287a8f2a1a3d52acb7a218e80"
server_key_iv: "1087d5643935c00716334e91a8dbf3b4"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.733374 20817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "395b0bed56f647f896b5f22d43baba6d"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "7c9070e287a8f2a1a3d52acb7a218e80"
server_key_iv: "1087d5643935c00716334e91a8dbf3b4"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.734670 20817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:48:39.735342 20832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.735532 20817 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:39.735616 20817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "395b0bed56f647f896b5f22d43baba6d"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "7c9070e287a8f2a1a3d52acb7a218e80"
server_key_iv: "1087d5643935c00716334e91a8dbf3b4"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.735670 20817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:39.745473 20817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:39.747279 20817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:39.747385 20817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:39.747561 20817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:39.747797 20817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:39.747826 20817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.747848 20817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:39.747859 20817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.753391 20817 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:43301
I20260806 06:48:39.753473 20952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:43301 every 8 connection(s)
I20260806 06:48:39.753759 20817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:39.762017 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20817
I20260806 06:48:39.762111 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:48:39.762356 6373 external_mini_cluster.cc:1428] Setting key 56ba5ac8ad82d88b89ff00e1500ba4aa
I20260806 06:48:39.763156 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.754453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:39739 (local address 127.6.57.126:46793)
0806 06:48:39.754553 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.754555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.754780 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:48:39.755287 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.755288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.755300 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.755353 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:39.756263 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.756753 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.757738 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.757950 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.758027 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.760180 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:39.760182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:39.760182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:39.760189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:39.761587 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.762017 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.762018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.762018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.762067 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.762413 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.762414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.762414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.762597 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:39.762687 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.762973 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.763049 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 06:48:39.763322 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.754433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:39739)
0806 06:48:39.754664 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.754698 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.755182 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.755419 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.755423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.755975 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:39.756164 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.756169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.756849 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.756850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.757614 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.757617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.758044 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.758045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.758166 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.758657 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:39.758664 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:39.760063 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:39.761675 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.761679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.761681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.761940 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.762158 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.762159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.762160 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.762327 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.762703 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:39.762711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:39.762847 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.762930 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.763097 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":137,"threads_started":1}
I20260806 06:48:39.763828 20953 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:39.763926 20953 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:39.764099 20953 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:39.764484 20611 ts_manager.cc:194] Registered new tserver with Master: 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:39.764866 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:39739
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:48:39.788673 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:39.872201 20960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:39.872390 20960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:39.872413 20960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:39.873456 20960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:39.873488 20960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:39.873514 20960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:39.874944 20960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:39.875131 20960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:39.875308 20960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:39.880185 20968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:39.880185 20966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.880389 20960 server_base.cc:1034] running on GCE node
W20260806 06:48:39.880213 20965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:39.880692 20960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:39.880972 20960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:39.882138 20960 hybrid_clock.cc:648] HybridClock initialized: now 1785998919882140 us; error 39 us; skew 500 ppm
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:39.884622 20960 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:39.884991 20960 webserver.cc:460] Webserver started at http://127.6.57.67:34985/ using document root <none> and password file <none>
I20260806 06:48:39.885190 20960 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:39.885229 20960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:39.885315 20960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:39.886276 20960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "54ac874d76454ff1899e91dc80bc1d93"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "a55b7101a6831539e24944f0e216ee0e"
server_key_iv: "615495aade28524dd7fc27144d557a82"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.886665 20960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "54ac874d76454ff1899e91dc80bc1d93"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "a55b7101a6831539e24944f0e216ee0e"
server_key_iv: "615495aade28524dd7fc27144d557a82"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.887931 20960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:39.888847 20975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.889083 20960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:39.889223 20960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "54ac874d76454ff1899e91dc80bc1d93"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "a55b7101a6831539e24944f0e216ee0e"
server_key_iv: "615495aade28524dd7fc27144d557a82"
server_key_version: "encryptionkey@0"
I20260806 06:48:39.889323 20960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:39.905354 20960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:39.907099 20960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:39.907213 20960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:39.907382 20960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:39.907684 20960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:39.907716 20960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.907737 20960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:39.907781 20960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:39.913347 20960 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:36907
I20260806 06:48:39.913441 21093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:36907 every 8 connection(s)
I20260806 06:48:39.913759 20960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:48:39 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998919, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:39.922570 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.914454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:40539 (local address 127.6.57.126:46793)
0806 06:48:39.914526 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.914528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.914735 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:48:39.915235 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.915236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.915249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.915304 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:39.916179 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.916672 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.917674 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.917916 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.917994 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.920035 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:39.920037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:39.920037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:39.920045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:39.921229 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.921608 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.921610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.921610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.921674 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.921921 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.921922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.921923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.922036 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:39.922114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.922375 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.922457 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:48:39.922755 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.914441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:40539)
0806 06:48:39.914622 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.914660 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.915127 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.915365 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.915370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.915900 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:39.916074 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.916078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.916772 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.916773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.917532 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.917534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.918001 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.918003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.918136 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.918660 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:39.918669 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:39.919936 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:39.921312 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.921319 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.921321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.921531 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.921743 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:39.921744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:39.921745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:39.921846 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:39.922109 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:39.922115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:39.922244 (+ 129us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.922341 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.922485 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":110,"threads_started":1}
I20260806 06:48:39.923252 21094 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:39.923352 21094 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:39.923525 21094 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:39.923933 20611 ts_manager.cc:194] Registered new tserver with Master: 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907)
I20260806 06:48:39.923964 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 20960
I20260806 06:48:39.924065 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:48:39.924278 6373 external_mini_cluster.cc:1428] Setting key 8f715b2b8ca93f13c8636edac83cc424
I20260806 06:48:39.924384 20611 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:40539
I20260806 06:48:39.925103 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:48:39.931643 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.926419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.6.57.126:46793)
0806 06:48:39.926527 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.926528 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.926592 (+ 64us) server_negotiation.cc:407] Connection header received
0806 06:48:39.926639 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.926640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.926649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.926694 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:39.927204 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.927632 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.928313 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.928480 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.928526 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.929537 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:39.929538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:39.929539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:39.929544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:39.930777 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.931063 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.931064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.931064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.931110 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:39.931266 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:39.931266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:39.931267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:39.931387 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:39.931456 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.931539 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.931589 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:48:39.933270 20611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:48:39.934880 20611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:48:39.941665 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43301 (local address 127.0.0.1:56706)
0806 06:48:39.937433 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.937443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.937504 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.937893 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.937896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.937899 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:39.937961 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.937963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.939331 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.939333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.940423 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.940426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941342 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.941422 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.941436 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.941533 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.941587 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":2,"thread_start_us":172,"threads_started":1}
I20260806 06:48:39.941699 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.6.57.66:43301)
0806 06:48:39.937648 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.937653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.937662 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:39.937759 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.937761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.937778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.937871 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:39.938035 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.939209 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.940512 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941243 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941322 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.941384 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.941557 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.941602 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260806 06:48:39.942001 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.6.57.67:36907)
0806 06:48:39.937852 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.937856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.937867 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:48:39.937905 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.937907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.937923 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.938006 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:39.938142 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.939255 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.940690 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941347 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941408 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.941452 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.941877 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.941926 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":82,"threads_started":1}
I20260806 06:48:39.942034 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43781 (local address 127.0.0.1:56982)
0806 06:48:39.937634 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.937646 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.937705 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.938772 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.938773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.938776 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:39.938815 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.938817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.940063 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.940065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.940990 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.940992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941831 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.941892 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.941896 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.941928 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.941968 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":89,"threads_started":1}
I20260806 06:48:39.942139 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:36907 (local address 127.0.0.1:33264)
0806 06:48:39.937268 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:39.937309 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:39.937596 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:39.938007 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:39.938008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:39.938010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:39.938072 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.938074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.939356 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.939358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.940599 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:39.940602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941493 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:39.941578 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.941582 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:39.941644 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:48:39.941688 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":45,"thread_start_us":73,"threads_started":1}
I20260806 06:48:39.942407 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:39.937621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.6.57.65:43781)
0806 06:48:39.938564 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:39.938568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:39.938587 (+ 19us) server_negotiation.cc:407] Connection header received
0806 06:48:39.938622 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:39.938624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:39.938655 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:39.938898 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:39.938962 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.939929 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.941073 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:39.941725 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:39.942166 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:39.942245 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:39.942278 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:48:39.942322 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":112,"threads_started":1}
I20260806 06:48:39.942531 20884 tablet_service.cc:1467] Processing CreateTablet for tablet d395b32528074b919a31fd6759a1b9b5 (DEFAULT_TABLE table=test-table [id=6742052b20484734a11123374e70660e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:39.942935 20884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d395b32528074b919a31fd6759a1b9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:39.945611 21109 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Bootstrap starting.
I20260806 06:48:39.946282 21109 tablet_bootstrap.cc:654] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:39.946260 20740 tablet_service.cc:1467] Processing CreateTablet for tablet d395b32528074b919a31fd6759a1b9b5 (DEFAULT_TABLE table=test-table [id=6742052b20484734a11123374e70660e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:39.946259 21025 tablet_service.cc:1467] Processing CreateTablet for tablet d395b32528074b919a31fd6759a1b9b5 (DEFAULT_TABLE table=test-table [id=6742052b20484734a11123374e70660e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:39.946616 20740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d395b32528074b919a31fd6759a1b9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:39.946626 21025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d395b32528074b919a31fd6759a1b9b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:39.946899 21109 log.cc:824] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:39.947616 21109 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: No bootstrap required, opened a new log
I20260806 06:48:39.947700 21109 ts_tablet_manager.cc:1397] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:39.948999 21109 raft_consensus.cc:348] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.949172 21109 raft_consensus.cc:374] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:39.949213 21109 raft_consensus.cc:729] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395b0bed56f647f896b5f22d43baba6d, State: Initialized, Role: FOLLOWER
I20260806 06:48:39.949328 21109 consensus_queue.cc:260] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.949599 21109 ts_tablet_manager.cc:1428] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:39.949744 20953 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
I20260806 06:48:39.949839 21112 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Bootstrap starting.
I20260806 06:48:39.950217 21113 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Bootstrap starting.
I20260806 06:48:39.950515 21112 tablet_bootstrap.cc:654] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:39.950922 21113 tablet_bootstrap.cc:654] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:39.951152 21112 log.cc:824] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:39.951565 21113 log.cc:824] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:39.952114 21112 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: No bootstrap required, opened a new log
I20260806 06:48:39.952203 21112 ts_tablet_manager.cc:1397] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:39.952416 21113 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: No bootstrap required, opened a new log
I20260806 06:48:39.952483 21113 ts_tablet_manager.cc:1397] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:48:39.953667 21112 raft_consensus.cc:348] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.953819 21112 raft_consensus.cc:374] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:39.953863 21112 raft_consensus.cc:729] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40736f190974f67a50611d283db3a90, State: Initialized, Role: FOLLOWER
I20260806 06:48:39.953970 21112 consensus_queue.cc:260] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.954041 21113 raft_consensus.cc:348] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.954222 21113 raft_consensus.cc:374] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:39.954247 20810 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
I20260806 06:48:39.954270 21113 raft_consensus.cc:729] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ac874d76454ff1899e91dc80bc1d93, State: Initialized, Role: FOLLOWER
I20260806 06:48:39.954384 21113 consensus_queue.cc:260] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:39.954469 21112 ts_tablet_manager.cc:1428] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:48:39.954713 21113 ts_tablet_manager.cc:1428] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:48:39.954836 21094 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
W20260806 06:48:40.004283 20955 tablet.cc:2369] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:48:40.026317 21117 raft_consensus.cc:482] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:40.026520 21117 raft_consensus.cc:504] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:40.026981 21117 leader_election.cc:288] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a40736f190974f67a50611d283db3a90 (127.6.57.65:43781), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:40.031065 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.027285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:43715 (local address 127.6.57.66:43301)
0806 06:48:40.027346 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.027349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.027360 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:48:40.027393 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.027394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.027408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.027522 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.027728 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.028508 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030027 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.030638 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030718 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.030727 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.030949 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.030992 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:48:40.031077 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.027189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43301 (local address 127.6.57.67:43715)
0806 06:48:40.027258 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.027270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.027312 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.027525 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.027526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.027528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.027620 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.027622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.028598 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.028598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.029920 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.029921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.030727 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.030841 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.030847 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.030920 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.030970 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":74}
I20260806 06:48:40.031098 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.027306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36981 (local address 127.6.57.65:43781)
0806 06:48:40.027350 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.027352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.027532 (+ 180us) server_negotiation.cc:407] Connection header received
0806 06:48:40.027569 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.027570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.027597 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.027647 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.027801 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.028510 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030027 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.030638 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030714 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.030727 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.030949 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.031028 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:48:40.031088 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.027236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43781 (local address 127.6.57.67:36981)
0806 06:48:40.027433 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.027440 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.027505 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.027667 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.027668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.027669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.027730 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.027731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.028597 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.028598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.029913 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.029917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.030728 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.030729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.030841 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.030847 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.030920 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.030970 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":80,"thread_start_us":134,"threads_started":1}
I20260806 06:48:40.031481 20762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d395b32528074b919a31fd6759a1b9b5" candidate_uuid: "54ac874d76454ff1899e91dc80bc1d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a40736f190974f67a50611d283db3a90" is_pre_election: true
I20260806 06:48:40.031477 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d395b32528074b919a31fd6759a1b9b5" candidate_uuid: "54ac874d76454ff1899e91dc80bc1d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395b0bed56f647f896b5f22d43baba6d" is_pre_election: true
I20260806 06:48:40.031630 20904 raft_consensus.cc:2454] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ac874d76454ff1899e91dc80bc1d93 in term 0.
I20260806 06:48:40.031635 20762 raft_consensus.cc:2454] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ac874d76454ff1899e91dc80bc1d93 in term 0.
I20260806 06:48:40.031898 20979 leader_election.cc:302] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ac874d76454ff1899e91dc80bc1d93, a40736f190974f67a50611d283db3a90; no voters:
I20260806 06:48:40.032081 21117 raft_consensus.cc:2781] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:48:40.032199 21117 raft_consensus.cc:482] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:48:40.032236 21117 raft_consensus.cc:3037] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.033032 21117 raft_consensus.cc:504] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:40.033196 21117 leader_election.cc:288] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [CANDIDATE]: Term 1 election: Requested vote from peers a40736f190974f67a50611d283db3a90 (127.6.57.65:43781), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:40.033311 20762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d395b32528074b919a31fd6759a1b9b5" candidate_uuid: "54ac874d76454ff1899e91dc80bc1d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a40736f190974f67a50611d283db3a90"
I20260806 06:48:40.033358 20904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d395b32528074b919a31fd6759a1b9b5" candidate_uuid: "54ac874d76454ff1899e91dc80bc1d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395b0bed56f647f896b5f22d43baba6d"
I20260806 06:48:40.033394 20762 raft_consensus.cc:3037] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.033423 20904 raft_consensus.cc:3037] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.034292 20904 raft_consensus.cc:2454] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ac874d76454ff1899e91dc80bc1d93 in term 1.
I20260806 06:48:40.034292 20762 raft_consensus.cc:2454] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ac874d76454ff1899e91dc80bc1d93 in term 1.
I20260806 06:48:40.034528 20978 leader_election.cc:302] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395b0bed56f647f896b5f22d43baba6d, 54ac874d76454ff1899e91dc80bc1d93; no voters:
I20260806 06:48:40.034680 21117 raft_consensus.cc:2781] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:40.034874 21117 raft_consensus.cc:686] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 LEADER]: Becoming Leader. State: Replica: 54ac874d76454ff1899e91dc80bc1d93, State: Running, Role: LEADER
I20260806 06:48:40.034997 21117 consensus_queue.cc:237] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:40.035938 20610 catalog_manager.cc:5495] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "54ac874d76454ff1899e91dc80bc1d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } health_report { overall_health: UNKNOWN } } }
I20260806 06:48:40.063843 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.061699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.6.57.67:36907)
0806 06:48:40.061761 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.061763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.061770 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:48:40.061819 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.061820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.061836 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.061891 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:48:40.061991 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.062420 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.063159 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.063333 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.063383 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.063521 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:48:40.063619 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:48:40.063677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.063714 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.063731 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:48:40.065646 20904 raft_consensus.cc:1261] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Refusing update from remote peer 54ac874d76454ff1899e91dc80bc1d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:40.065672 20762 raft_consensus.cc:1261] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Refusing update from remote peer 54ac874d76454ff1899e91dc80bc1d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:40.066037 21117 consensus_queue.cc:1035] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:40.066205 21119 consensus_queue.cc:1035] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:40.068718 21123 mvcc.cc:205] Tried to move back new op lower bound from 7315451576586178560 to 7315451576464117760. Current Snapshot: MvccSnapshot[applied={T|T < 7315451576586178560}]
I20260806 06:48:40.071435 20611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:40.071514 20611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:40.072278 20611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Sending DeleteTablet for 3 replicas of tablet d395b32528074b919a31fd6759a1b9b5
I20260806 06:48:40.072670 20884 tablet_service.cc:1514] Processing DeleteTablet for tablet d395b32528074b919a31fd6759a1b9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:56706
I20260806 06:48:40.072677 21025 tablet_service.cc:1514] Processing DeleteTablet for tablet d395b32528074b919a31fd6759a1b9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:33264
I20260806 06:48:40.072705 20740 tablet_service.cc:1514] Processing DeleteTablet for tablet d395b32528074b919a31fd6759a1b9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:56982
I20260806 06:48:40.072964 21132 tablet_replica.cc:330] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: stopping tablet replica
I20260806 06:48:40.072962 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20676
I20260806 06:48:40.072989 21133 tablet_replica.cc:330] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: stopping tablet replica
I20260806 06:48:40.074090 21133 raft_consensus.cc:2229] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:40.074251 21133 raft_consensus.cc:2258] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:40.074430 21131 tablet_replica.cc:330] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: stopping tablet replica
I20260806 06:48:40.074550 21131 raft_consensus.cc:2229] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:48:40.074646 21131 raft_consensus.cc:2258] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:40.075057 21131 ts_tablet_manager.cc:1905] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:40.076352 21133 ts_tablet_manager.cc:1905] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:40.077327 21131 ts_tablet_manager.cc:1918] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:40.077409 21131 log.cc:1196] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/d395b32528074b919a31fd6759a1b9b5
I20260806 06:48:40.077682 21131 ts_tablet_manager.cc:1939] T d395b32528074b919a31fd6759a1b9b5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting consensus metadata
I20260806 06:48:40.078063 20598 catalog_manager.cc:4841] TS 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): tablet d395b32528074b919a31fd6759a1b9b5 (table test-table [id=6742052b20484734a11123374e70660e]) successfully deleted
I20260806 06:48:40.078346 21133 ts_tablet_manager.cc:1918] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:40.078424 21133 log.cc:1196] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/d395b32528074b919a31fd6759a1b9b5
I20260806 06:48:40.078781 21133 ts_tablet_manager.cc:1939] T d395b32528074b919a31fd6759a1b9b5 P 395b0bed56f647f896b5f22d43baba6d: Deleting consensus metadata
I20260806 06:48:40.079198 20597 catalog_manager.cc:4841] TS 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): tablet d395b32528074b919a31fd6759a1b9b5 (table test-table [id=6742052b20484734a11123374e70660e]) successfully deleted
W20260806 06:48:40.080860 20598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:43781 (error 108)
I20260806 06:48:40.081072 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20817
I20260806 06:48:40.086345 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.086192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43781 (local address 127.0.0.1:56998)
0806 06:48:40.086254 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.086295 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:48:40.086412 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20960
W20260806 06:48:40.086469 20598 catalog_manager.cc:4584] TS a40736f190974f67a50611d283db3a90 (127.6.57.65:43781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d395b32528074b919a31fd6759a1b9b5: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43781: connect: Connection refused (error 111)
I20260806 06:48:40.092489 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 20579
I20260806 06:48:40.098694 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--webserver_interface=127.6.57.126
--webserver_port=44745
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:40.178704 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:40.178884 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:40.178906 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:40.180006 21135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:40.180044 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:40.180058 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:40.180095 21135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:40.180130 21135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:40.181552 21135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=44745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:40.181794 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:40.182027 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:48:40.190948 21135 server_base.cc:1034] running on GCE node
W20260806 06:48:40.190940 21141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.190922 21142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.190929 21144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.191376 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:40.191684 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:40.192848 21135 hybrid_clock.cc:648] HybridClock initialized: now 1785998920192838 us; error 39 us; skew 500 ppm
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:40.195390 21135 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:40.195864 21135 webserver.cc:460] Webserver started at http://127.6.57.126:44745/ using document root <none> and password file <none>
I20260806 06:48:40.196095 21135 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:40.196182 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:40.197480 21135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.198153 21151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.198328 21135 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:40.198434 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "53ca7ee193b042c7b50d854acd2bb32a"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "3c618e11e3dd60395b84ea8311701a15"
server_key_iv: "2ad7219851d4d96919e3dd710e998373"
server_key_version: "encryptionkey@0"
I20260806 06:48:40.198858 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:40.207588 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:40.209513 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:40.209669 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:40.213953 21135 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46793
I20260806 06:48:40.213985 21207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46793 every 8 connection(s)
I20260806 06:48:40.214397 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:40.216084 21208 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:48:40.216684 21208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap starting.
I20260806 06:48:40.223403 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21135
I20260806 06:48:40.223768 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:43781
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=36169
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:40.226112 21208 log.cc:824] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:40.227988 21208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:48:40.228298 21208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap complete.
I20260806 06:48:40.230638 21208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:40.230857 21208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Initialized, Role: FOLLOWER
I20260806 06:48:40.231016 21208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:40.231076 21208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:40.231127 21208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:40.231176 21208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:48:40.232124 21208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:40.232209 21208 leader_election.cc:302] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ca7ee193b042c7b50d854acd2bb32a; no voters:
I20260806 06:48:40.232416 21208 leader_election.cc:288] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:48:40.232450 21212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:48:40.232561 21212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 LEADER]: Becoming Leader. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Running, Role: LEADER
I20260806 06:48:40.232662 21212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:40.232688 21208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:40.232930 21214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:40.233034 21214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:40.232929 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ca7ee193b042c7b50d854acd2bb32a. Latest consensus state: current_term: 2 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:40.233409 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:40.233425 21216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:40.234256 21216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:40.234364 21216 catalog_manager.cc:1259] Loaded cluster ID: b14d4a08646c40bfb12a5a497ed48741
I20260806 06:48:40.234398 21216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:48:40.234659 21216 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 06:48:40.239688 21216 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 06:48:40.242339 21216 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:40.242561 21216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Loaded TSK: 0
I20260806 06:48:40.242774 21216 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:48:40.308907 21210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:40.309089 21210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:40.309118 21210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:40.310305 21210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:40.310345 21210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:40.310371 21210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:40.311791 21210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:43781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=36169
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:40.312009 21210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:40.312245 21210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:40.316869 21237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.316901 21238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.317111 21210 server_base.cc:1034] running on GCE node
W20260806 06:48:40.316867 21240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.317355 21210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:40.317584 21210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:40.318754 21210 hybrid_clock.cc:648] HybridClock initialized: now 1785998920318732 us; error 40 us; skew 500 ppm
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:40.321425 21210 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:40.321924 21210 webserver.cc:460] Webserver started at http://127.6.57.65:36169/ using document root <none> and password file <none>
I20260806 06:48:40.322191 21210 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:40.322268 21210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:40.323938 21210 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.324781 21247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.325107 21210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.325197 21210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "a40736f190974f67a50611d283db3a90"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "0260ba0769b4436659d178ecd4e326f9"
server_key_iv: "56cf1e9b0b7febff14f33128f67f16bf"
server_key_version: "encryptionkey@0"
I20260806 06:48:40.325603 21210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:40.349241 21210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:40.351544 21210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:40.351714 21210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:40.351955 21210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:40.352445 21254 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:48:40.353891 21210 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:48:40.353947 21210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:48:40.353979 21210 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:48:40.354851 21210 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:48:40.354893 21210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:40.354965 21254 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Bootstrap starting.
I20260806 06:48:40.362000 21210 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:43781
I20260806 06:48:40.362295 21366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:43781 every 8 connection(s)
I20260806 06:48:40.362414 21210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:40.365317 21254 log.cc:824] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:40.367729 21254 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:48:40.368052 21254 tablet_bootstrap.cc:492] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Bootstrap complete.
I20260806 06:48:40.369277 21254 ts_tablet_manager.cc:1397] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.011s
I20260806 06:48:40.370246 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21210
I20260806 06:48:40.370570 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:43301
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=40435
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:40.370699 21254 raft_consensus.cc:348] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:40.371019 21254 raft_consensus.cc:729] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40736f190974f67a50611d283db3a90, State: Initialized, Role: FOLLOWER
I20260806 06:48:40.371153 21254 consensus_queue.cc:260] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } }
I20260806 06:48:40.371438 21254 ts_tablet_manager.cc:1428] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:40.375602 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.363199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:35235)
0806 06:48:40.363584 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.363636 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.364374 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.364851 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.364855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.365276 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:40.365551 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.365558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.367133 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.367134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.368294 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.368298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.369198 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.369199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.369314 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.370079 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:40.370097 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:40.371954 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:40.373855 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.373862 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.373863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.374098 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.374404 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.374405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.374405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.374571 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.374949 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:40.374956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:40.375088 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.375200 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.375394 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":80,"threads_started":1}
I20260806 06:48:40.375648 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.363325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:35235 (local address 127.6.57.126:46793)
0806 06:48:40.363526 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.363535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.363757 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:48:40.364592 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.364598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.364649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.364778 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:40.365696 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.367006 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.368403 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.369083 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.369192 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.372086 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:40.372090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:40.372090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:40.372102 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:40.373779 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.374201 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.374204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.374204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.374328 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.374641 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.374642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.374643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.374870 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:40.374969 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.375346 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.375475 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260806 06:48:40.376392 21367 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:40.376502 21367 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:40.376727 21367 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:40.377574 21169 ts_manager.cc:194] Registered new tserver with Master: a40736f190974f67a50611d283db3a90 (127.6.57.65:43781)
I20260806 06:48:40.378916 21169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:35235
I20260806 06:48:40.379792 21367 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:48:40.386453 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.378253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43781 (local address 127.0.0.1:57006)
0806 06:48:40.378578 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.378607 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.378696 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.378906 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.378912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.379278 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:40.379335 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.379338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.380748 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.380749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.381338 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.381340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.381800 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.381802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.381880 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.382376 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:40.382380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:40.383715 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:40.385286 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.385293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.385295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.385518 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.385738 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.385739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.385739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.385798 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.386085 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:40.386091 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:40.386142 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.386243 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.386361 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":100,"threads_started":1}
I20260806 06:48:40.386454 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.378310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.6.57.65:43781)
0806 06:48:40.378465 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.378469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.378678 (+ 209us) server_negotiation.cc:407] Connection header received
0806 06:48:40.378798 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.378800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.378815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.378903 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:40.379941 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.380664 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.381440 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.381720 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.381799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.383804 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:40.383812 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:40.383812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:40.383819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:40.385196 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.385605 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.385608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.385608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.385668 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.385871 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.385872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.385872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.386010 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:40.386104 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.386280 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.386365 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260806 06:48:40.386857 21297 tablet_service.cc:1514] Processing DeleteTablet for tablet d395b32528074b919a31fd6759a1b9b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:57006
I20260806 06:48:40.387339 21376 tablet_replica.cc:330] stopping tablet replica
I20260806 06:48:40.387447 21376 raft_consensus.cc:2229] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:40.387511 21376 raft_consensus.cc:2258] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:40.387837 21376 ts_tablet_manager.cc:1905] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:40.390193 21376 ts_tablet_manager.cc:1918] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:40.390270 21376 log.cc:1196] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/d395b32528074b919a31fd6759a1b9b5
I20260806 06:48:40.390524 21376 ts_tablet_manager.cc:1939] T d395b32528074b919a31fd6759a1b9b5 P a40736f190974f67a50611d283db3a90: Deleting consensus metadata
I20260806 06:48:40.391053 21155 catalog_manager.cc:4841] TS a40736f190974f67a50611d283db3a90 (127.6.57.65:43781): tablet d395b32528074b919a31fd6759a1b9b5 (table test-table [id=6742052b20484734a11123374e70660e]) successfully deleted
W20260806 06:48:40.454846 21373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:40.455014 21373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:40.455034 21373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:40.456087 21373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:40.456125 21373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:40.456149 21373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:40.457684 21373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:43301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=40435
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:40.457954 21373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:40.458189 21373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:40.463022 21385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.463294 21373 server_base.cc:1034] running on GCE node
W20260806 06:48:40.463032 21383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.463052 21382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.463572 21373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:40.463841 21373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:40.464998 21373 hybrid_clock.cc:648] HybridClock initialized: now 1785998920464996 us; error 39 us; skew 500 ppm
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:40.467504 21373 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:40.467932 21373 webserver.cc:460] Webserver started at http://127.6.57.66:40435/ using document root <none> and password file <none>
I20260806 06:48:40.468185 21373 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:40.468268 21373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:40.469601 21373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.470209 21392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.470371 21373 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:40.470507 21373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "395b0bed56f647f896b5f22d43baba6d"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "7c9070e287a8f2a1a3d52acb7a218e80"
server_key_iv: "1087d5643935c00716334e91a8dbf3b4"
server_key_version: "encryptionkey@0"
I20260806 06:48:40.470827 21373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:40.497076 21373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:40.499536 21373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:40.499711 21373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:40.499949 21373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:40.500267 21373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:40.500321 21373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.500363 21373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:40.500391 21373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.506193 21373 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:43301
I20260806 06:48:40.506244 21508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:43301 every 8 connection(s)
I20260806 06:48:40.506644 21373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:48:40.507054 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21373
I20260806 06:48:40.507330 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:36907
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=34985
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:40.517746 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.507441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:42929 (local address 127.6.57.126:46793)
0806 06:48:40.507498 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.507500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.507707 (+ 207us) server_negotiation.cc:407] Connection header received
0806 06:48:40.508495 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.508496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.508511 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.508556 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:40.509709 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.510521 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.512148 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.512363 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.512417 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.514840 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:40.514842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:40.514842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:40.514856 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:40.516316 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.516738 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.516740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.516740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.516802 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.517062 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.517063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.517064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.517199 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:40.517274 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.517549 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.517641 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:48:40.517922 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.507398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:42929)
0806 06:48:40.507578 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.507627 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.508382 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.508645 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.508650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.509424 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:40.509611 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.509616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.510635 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.510636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.512045 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.512049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.512451 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.512453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.512614 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.513095 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:40.513106 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:40.514732 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:40.516417 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.516425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.516427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.516658 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.516874 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.516875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.516875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.516987 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.517277 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:40.517284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:40.517417 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.517516 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.517676 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260806 06:48:40.518491 21509 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:40.518603 21509 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:40.518806 21509 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:40.519204 21169 ts_manager.cc:194] Registered new tserver with Master: 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:40.519587 21169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:42929
W20260806 06:48:40.590688 21513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:40.590896 21513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:40.590934 21513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:40.592160 21513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:40.592221 21513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:40.592264 21513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:40.593786 21513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:36907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=34985
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:40.594033 21513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:40.594246 21513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:48:40.599145 21513 server_base.cc:1034] running on GCE node
W20260806 06:48:40.599145 21519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.599156 21518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:40.599153 21521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:40.599553 21513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:40.599795 21513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:40.600934 21513 hybrid_clock.cc:648] HybridClock initialized: now 1785998920600923 us; error 32 us; skew 500 ppm
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:40.603457 21513 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:40.603899 21513 webserver.cc:460] Webserver started at http://127.6.57.67:34985/ using document root <none> and password file <none>
I20260806 06:48:40.604110 21513 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:40.604162 21513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:40.605409 21513 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.605981 21530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.606154 21513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:40.606240 21513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "54ac874d76454ff1899e91dc80bc1d93"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "a55b7101a6831539e24944f0e216ee0e"
server_key_iv: "615495aade28524dd7fc27144d557a82"
server_key_version: "encryptionkey@0"
I20260806 06:48:40.606590 21513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:40.652705 21513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:40.655105 21513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:40.655225 21513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:40.655412 21513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:40.655767 21513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:40.655825 21513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.655861 21513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:40.655907 21513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.661572 21513 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:36907
I20260806 06:48:40.661629 21646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:36907 every 8 connection(s)
I20260806 06:48:40.661958 21513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:48:40.664099 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21513
Aug 06 06:48:40 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998920, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:40.672413 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.664488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.6.57.126:46793)
0806 06:48:40.664623 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.664625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.664634 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:40.664717 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.664717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.664726 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.664780 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:40.665455 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.665908 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.666966 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.667246 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.667293 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.669483 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:40.669484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:40.669485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:40.669491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:40.671205 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.671598 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.671599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.671600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.671667 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.671880 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.671881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.671882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.672003 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:40.672160 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.672216 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.672312 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260806 06:48:40.672794 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.662709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:50429 (local address 127.6.57.126:46793)
0806 06:48:40.662794 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.662797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.662981 (+ 184us) server_negotiation.cc:407] Connection header received
0806 06:48:40.663429 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.663430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.663441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.663491 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:40.664702 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.665498 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.666839 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.667097 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.667166 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.669797 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:40.669799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:40.669799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:40.669807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:40.671313 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.671751 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.671752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.671753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.671803 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:40.672067 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:40.672067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:40.672068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:40.672211 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:40.672272 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.672626 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.672699 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 06:48:40.672960 21649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.662667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:50429)
0806 06:48:40.662827 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.662862 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.663344 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.663541 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.663544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.664255 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:40.664549 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.664558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.665601 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.665602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.666742 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.666745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.667182 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.667183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.667309 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.667918 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:40.667927 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:40.669699 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:40.671389 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.671395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.671397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.671682 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.671873 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:40.671874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:40.671874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:40.671998 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:40.672279 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:40.672286 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:40.672459 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.672586 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.672766 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":100,"threads_started":1}
I20260806 06:48:40.673431 21647 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46793
I20260806 06:48:40.673527 21647 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:40.673717 21647 heartbeater.cc:511] Master 127.6.57.126:46793 requested a full tablet report, sending...
I20260806 06:48:40.674069 21169 ts_manager.cc:194] Registered new tserver with Master: 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907)
I20260806 06:48:40.674444 21169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:50429
I20260806 06:48:40.684317 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:48:40.686756 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.684663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.6.57.126:46793)
0806 06:48:40.684695 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.684697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.684779 (+ 82us) server_negotiation.cc:407] Connection header received
0806 06:48:40.684831 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.684832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.684860 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.684915 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:48:40.684994 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.685419 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.686110 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.686276 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.686324 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.686457 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:48:40.686609 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:48:40.686667 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.686700 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.686713 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:48:40.687019 21169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:48:40.688570 21169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:48:40.690680 21297 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 (DEFAULT_TABLE table=test-table [id=196da9aec9e94074b606db36dc0459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:40.690908 21297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdcc6da8aef4dcc88495c39b206e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:40.692322 21254 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Bootstrap starting.
I20260806 06:48:40.692626 21254 tablet_bootstrap.cc:654] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:40.693463 21254 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: No bootstrap required, opened a new log
I20260806 06:48:40.693516 21254 ts_tablet_manager.cc:1397] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:40.693661 21254 raft_consensus.cc:348] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.693709 21254 raft_consensus.cc:374] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:40.693738 21254 raft_consensus.cc:729] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40736f190974f67a50611d283db3a90, State: Initialized, Role: FOLLOWER
I20260806 06:48:40.693797 21254 consensus_queue.cc:260] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.693907 21254 ts_tablet_manager.cc:1428] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:40.694556 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.690298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:36907 (local address 127.0.0.1:33288)
0806 06:48:40.690344 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.690356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.690392 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.690875 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.690876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.690878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.690917 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.690918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.692230 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.692231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.693402 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.693404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.694286 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.694287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.694358 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.694380 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.694414 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.694461 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:48:40.694842 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.690359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43301 (local address 127.0.0.1:56708)
0806 06:48:40.690520 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.690531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.690572 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.691180 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.691182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.691184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.691221 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.691223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.692626 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.692627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.693652 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.693655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.694539 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.694540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.694633 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.694637 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.694697 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.694744 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260806 06:48:40.694887 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.690540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.6.57.67:36907)
0806 06:48:40.690711 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.690716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.690726 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:48:40.690773 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.690775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.690790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.690870 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.690996 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.692113 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.693498 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.694186 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.694672 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.694730 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.694772 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.694830 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260806 06:48:40.695137 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.690845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.6.57.66:43301)
0806 06:48:40.691014 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.691019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.691030 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:48:40.691066 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.691068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.691082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.691160 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.691298 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.692512 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.693741 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.694430 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.694531 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.694602 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.695020 (+ 418us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.695064 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260806 06:48:40.695552 21579 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 (DEFAULT_TABLE table=test-table [id=196da9aec9e94074b606db36dc0459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:40.695736 21442 tablet_service.cc:1467] Processing CreateTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 (DEFAULT_TABLE table=test-table [id=196da9aec9e94074b606db36dc0459eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:40.695865 21579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdcc6da8aef4dcc88495c39b206e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:40.696053 21442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fdcc6da8aef4dcc88495c39b206e0a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:40.698424 21657 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Bootstrap starting.
I20260806 06:48:40.698616 21658 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Bootstrap starting.
I20260806 06:48:40.699123 21657 tablet_bootstrap.cc:654] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:40.699220 21658 tablet_bootstrap.cc:654] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:40.699606 21657 log.cc:824] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:40.699622 21658 log.cc:824] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:40.700258 21658 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: No bootstrap required, opened a new log
I20260806 06:48:40.700279 21657 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: No bootstrap required, opened a new log
I20260806 06:48:40.700311 21658 ts_tablet_manager.cc:1397] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:48:40.700356 21657 ts_tablet_manager.cc:1397] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:48:40.701906 21658 raft_consensus.cc:348] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.701954 21657 raft_consensus.cc:348] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.702055 21658 raft_consensus.cc:374] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:40.702067 21657 raft_consensus.cc:374] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:40.702100 21658 raft_consensus.cc:729] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 395b0bed56f647f896b5f22d43baba6d, State: Initialized, Role: FOLLOWER
I20260806 06:48:40.702135 21657 raft_consensus.cc:729] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ac874d76454ff1899e91dc80bc1d93, State: Initialized, Role: FOLLOWER
I20260806 06:48:40.702215 21658 consensus_queue.cc:260] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.702237 21657 consensus_queue.cc:260] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.702582 21657 ts_tablet_manager.cc:1428] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:48:40.702603 21647 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
I20260806 06:48:40.702629 21509 heartbeater.cc:503] Master 127.6.57.126:46793 was elected leader, sending a full tablet report...
I20260806 06:48:40.702870 21658 ts_tablet_manager.cc:1428] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 06:48:40.757182 21511 tablet.cc:2369] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:48:40.863476 21368 tablet.cc:2369] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 06:48:40.912518 21648 tablet.cc:2369] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:48:40.961699 21372 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:40.961843 21372 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.962172 21372 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:40.966094 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.962553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:41185 (local address 127.6.57.67:36907)
0806 06:48:40.962595 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.962596 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.962609 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:48:40.962687 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.962688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.962701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.962777 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.962983 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.963710 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965143 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.965692 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965766 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.965777 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.965984 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.966029 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:48:40.966094 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.962458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:37391 (local address 127.6.57.66:43301)
0806 06:48:40.962537 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.962540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.962638 (+ 98us) server_negotiation.cc:407] Connection header received
0806 06:48:40.962687 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.962688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.962702 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.962777 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:40.962979 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.963725 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965143 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.965694 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965766 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.965776 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.965984 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.966025 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:48:40.966123 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.962363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43301 (local address 127.6.57.65:37391)
0806 06:48:40.962507 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.962520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.962586 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.962805 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.962806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.962809 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.962882 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.962885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.963806 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.963807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.965037 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.965041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.965786 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.965857 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.965886 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.965942 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.966012 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":116,"thread_start_us":52,"threads_started":1}
I20260806 06:48:40.966123 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.962363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:36907 (local address 127.6.57.65:41185)
0806 06:48:40.962508 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.962520 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:40.962587 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:40.962805 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:40.962807 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:40.962811 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:40.962911 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.962912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.963807 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.963808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.965037 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:40.965041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.965784 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.965785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:40.965857 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.965885 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 06:48:40.965942 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:48:40.966012 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":17,"thread_start_us":64,"threads_started":1}
I20260806 06:48:40.966560 21599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdcc6da8aef4dcc88495c39b206e0a5" candidate_uuid: "a40736f190974f67a50611d283db3a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ac874d76454ff1899e91dc80bc1d93" is_pre_election: true
I20260806 06:48:40.966729 21599 raft_consensus.cc:2454] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a40736f190974f67a50611d283db3a90 in term 0.
I20260806 06:48:40.966761 21463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdcc6da8aef4dcc88495c39b206e0a5" candidate_uuid: "a40736f190974f67a50611d283db3a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395b0bed56f647f896b5f22d43baba6d" is_pre_election: true
I20260806 06:48:40.966920 21463 raft_consensus.cc:2454] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a40736f190974f67a50611d283db3a90 in term 0.
I20260806 06:48:40.966952 21251 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ac874d76454ff1899e91dc80bc1d93, a40736f190974f67a50611d283db3a90; no voters:
I20260806 06:48:40.967149 21372 raft_consensus.cc:2781] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:48:40.967213 21372 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:48:40.967239 21372 raft_consensus.cc:3037] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.967916 21372 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.968065 21372 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 1 election: Requested vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
I20260806 06:48:40.968240 21463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdcc6da8aef4dcc88495c39b206e0a5" candidate_uuid: "a40736f190974f67a50611d283db3a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "395b0bed56f647f896b5f22d43baba6d"
I20260806 06:48:40.968248 21599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fdcc6da8aef4dcc88495c39b206e0a5" candidate_uuid: "a40736f190974f67a50611d283db3a90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ac874d76454ff1899e91dc80bc1d93"
I20260806 06:48:40.968315 21463 raft_consensus.cc:3037] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.968348 21599 raft_consensus.cc:3037] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:40.969025 21463 raft_consensus.cc:2454] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a40736f190974f67a50611d283db3a90 in term 1.
I20260806 06:48:40.969141 21599 raft_consensus.cc:2454] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a40736f190974f67a50611d283db3a90 in term 1.
I20260806 06:48:40.969198 21250 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 395b0bed56f647f896b5f22d43baba6d, a40736f190974f67a50611d283db3a90; no voters:
I20260806 06:48:40.969278 21372 raft_consensus.cc:2781] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:40.969422 21372 raft_consensus.cc:686] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 LEADER]: Becoming Leader. State: Replica: a40736f190974f67a50611d283db3a90, State: Running, Role: LEADER
I20260806 06:48:40.969492 21372 consensus_queue.cc:237] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:40.970337 21169 catalog_manager.cc:5495] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 reported cstate change: term changed from 0 to 1, leader changed from <none> to a40736f190974f67a50611d283db3a90 (127.6.57.65). New cstate: current_term: 1 leader_uuid: "a40736f190974f67a50611d283db3a90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } health_report { overall_health: HEALTHY } } }
I20260806 06:48:40.978768 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.976474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.6.57.65:43781)
0806 06:48:40.976628 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:40.976630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:40.976637 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:48:40.976664 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:40.976665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:40.976680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:40.976756 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:48:40.976846 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.977318 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.978011 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:40.978226 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:40.978280 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:40.978395 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:48:40.978538 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:48:40.978609 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:40.978643 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:48:40.978660 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260806 06:48:40.980124 21463 raft_consensus.cc:1261] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Refusing update from remote peer a40736f190974f67a50611d283db3a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:40.980152 21599 raft_consensus.cc:1261] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Refusing update from remote peer a40736f190974f67a50611d283db3a90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:40.980432 21665 consensus_queue.cc:1035] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:40.980571 21372 consensus_queue.cc:1035] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:40.983609 21671 mvcc.cc:205] Tried to move back new op lower bound from 7315451580332593152 to 7315451580291776512. Current Snapshot: MvccSnapshot[applied={T|T < 7315451580332593152}]
I20260806 06:48:40.984705 21672 mvcc.cc:205] Tried to move back new op lower bound from 7315451580332593152 to 7315451580291776512. Current Snapshot: MvccSnapshot[applied={T|T < 7315451580332593152}]
I20260806 06:48:40.984731 21169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:40.984809 21169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:40.985605 21169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Sending DeleteTablet for 3 replicas of tablet 2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:40.985903 21442 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:56708
I20260806 06:48:40.985987 21579 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:33288
I20260806 06:48:40.986069 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21210
I20260806 06:48:40.986094 21297 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:57006
I20260806 06:48:40.986130 21679 tablet_replica.cc:330] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: stopping tablet replica
I20260806 06:48:40.986192 21578 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:40 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:33288
I20260806 06:48:40.986224 21679 raft_consensus.cc:2229] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:40.986315 21679 raft_consensus.cc:2258] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:40.986946 21679 ts_tablet_manager.cc:1905] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:40.987221 21680 tablet_replica.cc:330] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: stopping tablet replica
I20260806 06:48:40.987306 21680 raft_consensus.cc:2229] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:40.987394 21680 raft_consensus.cc:2258] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:40.987824 21680 ts_tablet_manager.cc:1905] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:40.988814 21679 ts_tablet_manager.cc:1918] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:40.988879 21679 log.cc:1196] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:40.989148 21679 ts_tablet_manager.cc:1939] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 395b0bed56f647f896b5f22d43baba6d: Deleting consensus metadata
I20260806 06:48:40.989471 21680 ts_tablet_manager.cc:1918] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:40.989547 21680 log.cc:1196] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:40.989594 21154 catalog_manager.cc:4841] TS 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): tablet 2fdcc6da8aef4dcc88495c39b206e0a5 (table test-table [id=196da9aec9e94074b606db36dc0459eb]) successfully deleted
I20260806 06:48:40.989838 21680 ts_tablet_manager.cc:1939] T 2fdcc6da8aef4dcc88495c39b206e0a5 P 54ac874d76454ff1899e91dc80bc1d93: Deleting consensus metadata
I20260806 06:48:40.990240 21155 catalog_manager.cc:4841] TS 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): tablet 2fdcc6da8aef4dcc88495c39b206e0a5 (table test-table [id=196da9aec9e94074b606db36dc0459eb]) successfully deleted
W20260806 06:48:40.990374 21155 catalog_manager.cc:4808] TS 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): delete failed for tablet 2fdcc6da8aef4dcc88495c39b206e0a5 because the tablet was not found. No further retry: Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
W20260806 06:48:40.993458 21155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:43781 (error 108)
I20260806 06:48:40.993973 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21373
I20260806 06:48:40.993984 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:40.993851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43781 (local address 127.0.0.1:57036)
0806 06:48:40.993907 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:40.993960 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:48:40.994163 21155 catalog_manager.cc:4584] TS a40736f190974f67a50611d283db3a90 (127.6.57.65:43781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fdcc6da8aef4dcc88495c39b206e0a5: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43781: connect: Connection refused (error 111)
I20260806 06:48:41.000298 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21513
I20260806 06:48:41.005407 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21135
I20260806 06:48:41.011415 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--webserver_interface=127.6.57.126
--webserver_port=44745
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:41.091849 21682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:41.092025 21682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:41.092046 21682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:41.093057 21682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:41.093089 21682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:41.093103 21682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:41.093115 21682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:41.093127 21682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:41.094446 21682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=44745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:41.094734 21682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:41.094957 21682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:41.099743 21691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.099859 21688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.099963 21682 server_base.cc:1034] running on GCE node
W20260806 06:48:41.099874 21689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.100253 21682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:41.100564 21682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:41.101734 21682 hybrid_clock.cc:648] HybridClock initialized: now 1785998921101721 us; error 41 us; skew 500 ppm
Aug 06 06:48:41 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:41.104254 21682 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:41.104707 21682 webserver.cc:460] Webserver started at http://127.6.57.126:44745/ using document root <none> and password file <none>
I20260806 06:48:41.104925 21682 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:41.104988 21682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:41.106328 21682 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:41.107108 21698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.107296 21682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:48:41.107396 21682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "53ca7ee193b042c7b50d854acd2bb32a"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "3c618e11e3dd60395b84ea8311701a15"
server_key_iv: "2ad7219851d4d96919e3dd710e998373"
server_key_version: "encryptionkey@0"
I20260806 06:48:41.107713 21682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:41.126677 21682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:41.128904 21682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:41.129067 21682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:41.133229 21682 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46793
I20260806 06:48:41.133363 21751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46793 every 8 connection(s)
I20260806 06:48:41.133692 21682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:41.135247 21752 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:48:41.135892 21752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap starting.
I20260806 06:48:41.136564 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21682
I20260806 06:48:41.136839 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:43781
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=36169
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:41.144477 21752 log.cc:824] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:41.146474 21752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:48:41.146816 21752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a: Bootstrap complete.
I20260806 06:48:41.149005 21752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:41.149253 21752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Initialized, Role: FOLLOWER
I20260806 06:48:41.149426 21752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:41.149502 21752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:41.149562 21752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:41.149613 21752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:48:41.150571 21752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:41.150686 21752 leader_election.cc:302] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ca7ee193b042c7b50d854acd2bb32a; no voters:
I20260806 06:48:41.150876 21752 leader_election.cc:288] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:48:41.150909 21756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:48:41.151044 21756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [term 3 LEADER]: Becoming Leader. State: Replica: 53ca7ee193b042c7b50d854acd2bb32a, State: Running, Role: LEADER
I20260806 06:48:41.151149 21752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:41.151157 21756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } }
I20260806 06:48:41.151456 21756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ca7ee193b042c7b50d854acd2bb32a. Latest consensus state: current_term: 3 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:41.151517 21756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:41.151676 21758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "53ca7ee193b042c7b50d854acd2bb32a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca7ee193b042c7b50d854acd2bb32a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46793 } } }
I20260806 06:48:41.151758 21758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca7ee193b042c7b50d854acd2bb32a [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:41.151859 21760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:41.152814 21760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:41.152917 21760 catalog_manager.cc:1259] Loaded cluster ID: b14d4a08646c40bfb12a5a497ed48741
I20260806 06:48:41.152951 21760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 06:48:41.153445 21760 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdcb0b849ad google::LogMessage::Fail() at ??:0
@ 0x7fdcb0b8788a google::LogMessage::SendToLog() at ??:0
@ 0x7fdcb0b844de google::LogMessage::Flush() at ??:0
@ 0x7fdcb0b85a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdcb8aa50f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdcb8aae8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdcb2a520d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdcb2a47954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdcb38ab6db start_thread at ??:0
@ 0x7fdcaf68261f clone at ??:0
W20260806 06:48:41.229686 21754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:41.229897 21754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:41.229934 21754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:41.231115 21754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:41.231177 21754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:41.231222 21754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:41.232764 21754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:43781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=36169
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:41.233013 21754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:41.233232 21754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:41.237795 21792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.237803 21790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.238163 21754 server_base.cc:1034] running on GCE node
W20260806 06:48:41.237804 21789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.238449 21754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:41.238732 21754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:41.239902 21754 hybrid_clock.cc:648] HybridClock initialized: now 1785998921239885 us; error 31 us; skew 500 ppm
Aug 06 06:48:41 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:41.243024 21754 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:41.243500 21754 webserver.cc:460] Webserver started at http://127.6.57.65:36169/ using document root <none> and password file <none>
I20260806 06:48:41.243768 21754 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:41.243839 21754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:41.245182 21754 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:41.245960 21799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.246196 21754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:41.246326 21754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "a40736f190974f67a50611d283db3a90"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "0260ba0769b4436659d178ecd4e326f9"
server_key_iv: "56cf1e9b0b7febff14f33128f67f16bf"
server_key_version: "encryptionkey@0"
I20260806 06:48:41.246728 21754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:41.263237 21754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:41.265424 21754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:41.265584 21754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:41.265802 21754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:41.266227 21806 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:48:41.268169 21754 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:48:41.268240 21754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:48:41.268281 21754 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:48:41.268883 21754 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:48:41.268940 21754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:48:41.269008 21806 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Bootstrap starting.
I20260806 06:48:41.276015 21754 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:43781
I20260806 06:48:41.276129 21915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:43781 every 8 connection(s)
I20260806 06:48:41.276422 21754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:41.280011 21806 log.cc:824] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:41.282683 21806 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:48:41.282754 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21754
I20260806 06:48:41.282992 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:43301
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=40435
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:48:41.283006 21806 tablet_bootstrap.cc:492] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Bootstrap complete.
I20260806 06:48:41.284214 21806 ts_tablet_manager.cc:1397] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260806 06:48:41.285579 21806 raft_consensus.cc:348] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:41.285887 21806 raft_consensus.cc:729] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40736f190974f67a50611d283db3a90, State: Initialized, Role: FOLLOWER
I20260806 06:48:41.286039 21806 consensus_queue.cc:260] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:41.286311 21806 ts_tablet_manager.cc:1428] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260806 06:48:41.342253 21918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:48:41.277152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:43535)
0806 06:48:41.277317 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.277358 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:41.277840 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:41.342041 (+ 64201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
W20260806 06:48:41.342527 21916 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:46793 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:46793: Client connection negotiation failed: client connection to 127.6.57.126:46793: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:48:41.342911 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.342789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:53227)
0806 06:48:41.342849 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.342891 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:48:41.343298 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.343203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:38779)
0806 06:48:41.343251 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.343275 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:48:41.343408 21916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:48:41.367035 21920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:41.367268 21920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:41.367304 21920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:41.368459 21920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:41.368522 21920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:41.368580 21920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:41.369966 21920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:43301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=40435
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:41.370218 21920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:41.370420 21920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:41.375205 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.375185 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.375177 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.375245 21920 server_base.cc:1034] running on GCE node
I20260806 06:48:41.375501 21920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:41.375730 21920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:41.376873 21920 hybrid_clock.cc:648] HybridClock initialized: now 1785998921376860 us; error 34 us; skew 500 ppm
Aug 06 06:48:41 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:41.379349 21920 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:41.379801 21920 webserver.cc:460] Webserver started at http://127.6.57.66:40435/ using document root <none> and password file <none>
I20260806 06:48:41.380021 21920 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:41.380095 21920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:41.381399 21920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:41.381982 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.382166 21920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:41.382272 21920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "395b0bed56f647f896b5f22d43baba6d"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "7c9070e287a8f2a1a3d52acb7a218e80"
server_key_iv: "1087d5643935c00716334e91a8dbf3b4"
server_key_version: "encryptionkey@0"
I20260806 06:48:41.382664 21920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:41.391422 21920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:41.393668 21920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:41.393821 21920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:41.394037 21920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:41.394361 21920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:41.394416 21920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.394454 21920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:41.394512 21920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.400643 21920 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:43301
I20260806 06:48:41.400694 22054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:43301 every 8 connection(s)
I20260806 06:48:41.401003 21920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:48:41.401932 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.401582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:41447)
0806 06:48:41.401738 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.401811 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260806 06:48:41.402318 22055 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:46793 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:46793: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
I20260806 06:48:41.402601 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.402523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:44771)
0806 06:48:41.402559 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.402584 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:48:41.402833 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.402784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:39105)
0806 06:48:41.402804 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.402822 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260806 06:48:41.402932 22055 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:48:41.408807 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 21920
I20260806 06:48:41.409021 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:36907
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:46793
--webserver_port=34985
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:41.490072 22058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:41.490276 22058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:41.490299 22058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:41.491432 22058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:41.491474 22058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:41.491500 22058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:41.492862 22058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:36907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=34985
--tserver_master_addrs=127.6.57.126:46793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:41.493091 22058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:41.493314 22058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:41.498276 22063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.498286 22064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:41.498288 22066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:41.498545 22058 server_base.cc:1034] running on GCE node
I20260806 06:48:41.498740 22058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:41.498972 22058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:41.500150 22058 hybrid_clock.cc:648] HybridClock initialized: now 1785998921500136 us; error 27 us; skew 500 ppm
Aug 06 06:48:41 dist-test-slave-5l3k krb5kdc[20566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:41.502458 22058 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:41.502954 22058 webserver.cc:460] Webserver started at http://127.6.57.67:34985/ using document root <none> and password file <none>
I20260806 06:48:41.503134 22058 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:41.503175 22058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:41.504449 22058 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:41.504992 22073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.505168 22058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.505251 22058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "54ac874d76454ff1899e91dc80bc1d93"
format_stamp: "Formatted at 2026-08-06 06:48:39 on dist-test-slave-5l3k"
server_key: "a55b7101a6831539e24944f0e216ee0e"
server_key_iv: "615495aade28524dd7fc27144d557a82"
server_key_version: "encryptionkey@0"
I20260806 06:48:41.505537 22058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:41.514756 22058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:41.516408 22058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:41.516515 22058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:41.516683 22058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:41.516973 22058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:41.517007 22058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.517026 22058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:41.517079 22058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:41.522590 22058 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:36907
I20260806 06:48:41.522689 22192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:36907 every 8 connection(s)
I20260806 06:48:41.522984 22058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:48:41.523947 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.523623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:57197)
0806 06:48:41.523757 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.523820 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260806 06:48:41.524293 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22058
W20260806 06:48:41.524335 22193 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:46793 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:46793: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
I20260806 06:48:41.524614 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.524528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:44901)
0806 06:48:41.524562 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.524588 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 06:48:41.524964 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:41.524892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:34941)
0806 06:48:41.524921 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:41.524943 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 06:48:41.525103 22193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:48:42.343945 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.343701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:59375)
0806 06:48:42.343832 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:42.343883 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260806 06:48:42.403432 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.403223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:58061)
0806 06:48:42.403347 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:42.403378 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
I20260806 06:48:42.525673 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.525416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:49451)
0806 06:48:42.525549 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:42.525626 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260806 06:48:42.716507 22200 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:42.716645 22200 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:42.717010 22200 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
Aug 06 06:48:42 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.66@KRBTEST.COM
Aug 06 06:48:42 dist-test-slave-5l3k krb5kdc[20566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998921, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.67@KRBTEST.COM
I20260806 06:48:42.727742 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.717136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:36907 (local address 127.6.57.65:53015)
0806 06:48:42.717343 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:42.717353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:42.717411 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:42.718204 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:42.718207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:42.718658 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:42.718850 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:42.718856 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.720012 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.720019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:42.720781 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:42.720784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.721396 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.721397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:42.721473 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:42.722048 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:42.722057 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:42.724080 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:42.726122 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:42.726130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:42.726367 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:42.726686 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726686 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:42.726687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:42.726812 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:42.727210 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:42.727217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:42.727345 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 06:48:42.727450 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 06:48:42.727635 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":167,"thread_start_us":62,"threads_started":1}
I20260806 06:48:42.727761 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.717249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:53015 (local address 127.6.57.67:36907)
0806 06:48:42.717445 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:42.717454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:42.717494 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:48:42.717957 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:42.717959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:42.718004 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:42.718177 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:42.718980 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.719870 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.720896 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.721292 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.721426 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:42.724185 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:42.724191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:42.724191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:42.724206 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:42.726012 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726467 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:42.726470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:42.726470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:42.726585 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726891 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:42.726892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:42.726892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:42.727127 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:42.727216 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:42.727505 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 06:48:42.727634 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":52,"threads_started":1}
I20260806 06:48:42.727815 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.717295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:39049 (local address 127.6.57.66:43301)
0806 06:48:42.717439 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:42.717452 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:42.717490 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:48:42.717956 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:42.717959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:42.718009 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:42.718176 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:42.718968 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.719850 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.720869 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.721254 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.721375 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:42.723761 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:42.723765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:42.723766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:42.723785 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:42.726025 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726557 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:42.726560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:42.726560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:42.726678 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726922 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:42.726923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:42.726923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:42.727161 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:42.727267 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:42.727533 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 06:48:42.727689 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260806 06:48:42.728540 22197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:42.717136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43301 (local address 127.6.57.65:39049)
0806 06:48:42.717209 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:42.717223 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:42.717296 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:42.718204 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:42.718207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:42.718655 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:42.718849 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:42.718855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.719972 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.719973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:42.720780 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:42.720782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:42.721347 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:42.721348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:42.721469 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:42.722081 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:42.722085 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:42.723643 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:42.726140 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:42.726141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:42.726429 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:42.726760 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:42.726761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:42.726761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:42.726838 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:42.727266 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:42.727267 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:42.727362 (+ 95us) client_negotiation.cc:769] Sending connection context
0806 06:48:42.727450 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:48:42.727729 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":142}
W20260806 06:48:42.728641 21803 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
W20260806 06:48:42.729256 21802 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:42.729321 21802 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a40736f190974f67a50611d283db3a90; no voters: 395b0bed56f647f896b5f22d43baba6d, 54ac874d76454ff1899e91dc80bc1d93
I20260806 06:48:42.729454 22200 raft_consensus.cc:2726] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:48:43.344660 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:43.344454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:36611)
0806 06:48:43.344578 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:43.344614 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 06:48:43.404047 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:43.403853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:32857)
0806 06:48:43.403964 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:43.404004 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260806 06:48:43.526438 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:43.526194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:58553)
0806 06:48:43.526363 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:43.526401 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260806 06:48:44.345335 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:44.345113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:48621)
0806 06:48:44.345255 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:44.345286 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260806 06:48:44.404682 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:44.404482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:43191)
0806 06:48:44.404607 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:44.404645 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260806 06:48:44.527163 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:44.526935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:56017)
0806 06:48:44.527089 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:44.527124 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260806 06:48:44.841606 22210 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:44.841696 22210 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:44.841884 22210 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
W20260806 06:48:44.842345 21802 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
W20260806 06:48:44.842406 21803 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:44.842433 21802 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a40736f190974f67a50611d283db3a90; no voters: 395b0bed56f647f896b5f22d43baba6d, 54ac874d76454ff1899e91dc80bc1d93
I20260806 06:48:44.842556 22210 raft_consensus.cc:2726] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:48:45.345999 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:45.345731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:57163)
0806 06:48:45.345894 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:45.345936 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260806 06:48:45.405457 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:45.405111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:58951)
0806 06:48:45.405385 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:45.405420 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":77,"threads_started":1}
I20260806 06:48:45.527828 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:45.527611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:49933)
0806 06:48:45.527755 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:45.527791 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260806 06:48:46.346870 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:46.346644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:56925)
0806 06:48:46.346785 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:46.346824 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260806 06:48:46.406208 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:46.405958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:45061)
0806 06:48:46.406125 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:46.406170 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260806 06:48:46.528466 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:46.528264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:53267)
0806 06:48:46.528394 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:46.528427 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260806 06:48:47.347685 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:47.347419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:58437)
0806 06:48:47.347592 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:47.347638 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260806 06:48:47.406881 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:47.406675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:48979)
0806 06:48:47.406813 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:47.406846 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260806 06:48:47.529161 22219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:47.528941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:39185)
0806 06:48:47.529083 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:47.529123 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260806 06:48:48.049575 22220 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:48.049685 22220 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:48.049896 22220 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
W20260806 06:48:48.050355 21802 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
W20260806 06:48:48.050408 21803 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:48.050439 21802 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a40736f190974f67a50611d283db3a90; no voters: 395b0bed56f647f896b5f22d43baba6d, 54ac874d76454ff1899e91dc80bc1d93
I20260806 06:48:48.050552 22220 raft_consensus.cc:2726] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:48:48.348477 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:48.348236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:49331)
0806 06:48:48.348381 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:48.348420 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260806 06:48:48.407547 22222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:48.407312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:37993)
0806 06:48:48.407447 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:48.407481 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260806 06:48:48.529774 22223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:48.529562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:44791)
0806 06:48:48.529694 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:48.529731 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260806 06:48:49.349144 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:49.348928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:42599)
0806 06:48:49.349058 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:49.349095 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260806 06:48:49.408165 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:49.407978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:59745)
0806 06:48:49.408094 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:49.408126 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260806 06:48:49.530527 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:49.530259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:36111)
0806 06:48:49.530409 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:49.530438 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 06:48:50.350036 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:50.349786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:49403)
0806 06:48:50.349935 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:50.349985 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260806 06:48:50.408877 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:50.408637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:47457)
0806 06:48:50.408780 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:50.408838 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260806 06:48:50.531280 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:50.531037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:33139)
0806 06:48:50.531195 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:50.531238 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260806 06:48:51.350893 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:51.350671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:50493)
0806 06:48:51.350811 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:51.350846 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260806 06:48:51.409533 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:51.409329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:50029)
0806 06:48:51.409456 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:51.409490 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260806 06:48:51.415486 22232 raft_consensus.cc:482] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:51.415560 22232 raft_consensus.cc:504] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54ac874d76454ff1899e91dc80bc1d93" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 36907 } } peers { permanent_uuid: "395b0bed56f647f896b5f22d43baba6d" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43301 } } peers { permanent_uuid: "a40736f190974f67a50611d283db3a90" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43781 } }
I20260806 06:48:51.415714 22232 leader_election.cc:288] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907), 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301)
W20260806 06:48:51.416126 21802 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 395b0bed56f647f896b5f22d43baba6d (127.6.57.66:43301): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
W20260806 06:48:51.416198 21803 leader_election.cc:341] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 54ac874d76454ff1899e91dc80bc1d93 (127.6.57.67:36907): Not found: Tablet not found: 2fdcc6da8aef4dcc88495c39b206e0a5
I20260806 06:48:51.416245 21803 leader_election.cc:302] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a40736f190974f67a50611d283db3a90; no voters: 395b0bed56f647f896b5f22d43baba6d, 54ac874d76454ff1899e91dc80bc1d93
I20260806 06:48:51.416342 22232 raft_consensus.cc:2726] T 2fdcc6da8aef4dcc88495c39b206e0a5 P a40736f190974f67a50611d283db3a90 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:48:51.531946 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:51.531732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:47647)
0806 06:48:51.531871 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:51.531906 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260806 06:48:52.351539 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:52.351332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:53201)
0806 06:48:52.351458 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:52.351493 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260806 06:48:52.410192 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:52.409991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:41917)
0806 06:48:52.410120 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:52.410154 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260806 06:48:52.532630 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:52.532405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:34219)
0806 06:48:52.532556 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:52.532591 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260806 06:48:53.352202 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:53.351999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:51771)
0806 06:48:53.352123 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:53.352157 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260806 06:48:53.410871 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:53.410646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:60185)
0806 06:48:53.410794 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:53.410829 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260806 06:48:53.533418 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:53.533127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:56643)
0806 06:48:53.533341 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:53.533376 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":79,"threads_started":1}
I20260806 06:48:54.352839 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:54.352649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:41487)
0806 06:48:54.352761 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:54.352792 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260806 06:48:54.411485 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:54.411259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:57625)
0806 06:48:54.411398 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:54.411443 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
I20260806 06:48:54.534183 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:54.533965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:49285)
0806 06:48:54.534104 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:54.534140 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260806 06:48:55.353479 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:55.353274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:40375)
0806 06:48:55.353399 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:55.353431 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260806 06:48:55.412194 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:55.411977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:60111)
0806 06:48:55.412111 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:55.412149 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260806 06:48:55.534988 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:55.534783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:53385)
0806 06:48:55.534911 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:55.534944 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260806 06:48:56.354197 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:56.353988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.65:59403)
0806 06:48:56.354120 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:56.354153 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260806 06:48:56.412886 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:56.412696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.66:59929)
0806 06:48:56.412817 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:56.412848 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260806 06:48:56.532769 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21754
I20260806 06:48:56.535724 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:56.535537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46793 (local address 127.6.57.67:48047)
0806 06:48:56.535652 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:56.535685 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":64,"threads_started":1}
I20260806 06:48:56.540060 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 21920
I20260806 06:48:56.545745 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22058
2026-08-06T06:48:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18494 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:48:56 dist-test-slave-5l3k krb5kdc[22252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:48:56 dist-test-slave-5l3k krb5kdc[22252](info): set up 2 sockets
Aug 06 06:48:56 dist-test-slave-5l3k krb5kdc[22252](info): commencing operation
krb5kdc: starting...
W20260806 06:48:57.613372 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:48:57 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:48:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:48:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:48:57.733055 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:57.817058 22265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:57.817283 22265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:57.817322 22265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:57.818536 22265 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:48:57.818591 22265 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:48:57.818619 22265 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:48:57.818648 22265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:48:57.818676 22265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:57.818707 22265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:57.818744 22265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:48:57.818768 22265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:48:57.820268 22265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:57.820549 22265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:57.820792 22265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:48:57.825990 22265 server_base.cc:1034] running on GCE node
W20260806 06:48:57.825954 22270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:57.826001 22271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:57.825961 22273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:57.826643 22265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:57.826946 22265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:57.828112 22265 hybrid_clock.cc:648] HybridClock initialized: now 1785998937828104 us; error 37 us; skew 500 ppm
Aug 06 06:48:57 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998937, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:57.830554 22265 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:48:57.831022 22265 webserver.cc:460] Webserver started at http://127.6.57.126:41447/ using document root <none> and password file <none>
I20260806 06:48:57.831287 22265 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:57.831357 22265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:57.831511 22265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:57.832561 22265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "7616b60c69fd467a848c5d8fed919e3d"
format_stamp: "Formatted at 2026-08-06 06:48:57 on dist-test-slave-5l3k"
server_key: "d01edd62daab47905b0919bb987582f6"
server_key_iv: "4559bc5e75759678323fce81115d6ecb"
server_key_version: "encryptionkey@0"
I20260806 06:48:57.832921 22265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "7616b60c69fd467a848c5d8fed919e3d"
format_stamp: "Formatted at 2026-08-06 06:48:57 on dist-test-slave-5l3k"
server_key: "d01edd62daab47905b0919bb987582f6"
server_key_iv: "4559bc5e75759678323fce81115d6ecb"
server_key_version: "encryptionkey@0"
I20260806 06:48:57.834239 22265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:48:57.835005 22282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:57.835167 22265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:57.835268 22265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "7616b60c69fd467a848c5d8fed919e3d"
format_stamp: "Formatted at 2026-08-06 06:48:57 on dist-test-slave-5l3k"
server_key: "d01edd62daab47905b0919bb987582f6"
server_key_iv: "4559bc5e75759678323fce81115d6ecb"
server_key_version: "encryptionkey@0"
I20260806 06:48:57.835355 22265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:57.844878 22265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:57.847029 22265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:57.847201 22265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:57.851480 22265 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41151
I20260806 06:48:57.851516 22336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41151 every 8 connection(s)
I20260806 06:48:57.851863 22265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:48:57.852819 22337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:57.855424 22337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap starting.
I20260806 06:48:57.856031 22337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:57.856321 22337 log.cc:824] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:57.857044 22337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: No bootstrap required, opened a new log
I20260806 06:48:57.857372 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22265
I20260806 06:48:57.857504 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:48:57.857789 6373 external_mini_cluster.cc:1428] Setting key fa34f748f0816dba71233391b25fa8dc
I20260806 06:48:57.859579 22337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:48:57.859764 22337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:57.859829 22337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Initialized, Role: FOLLOWER
I20260806 06:48:57.860028 22337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:48:57.860141 22337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:48:57.860198 22337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:48:57.860263 22337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:57.861037 22337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:48:57.861135 22337 leader_election.cc:302] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7616b60c69fd467a848c5d8fed919e3d; no voters:
I20260806 06:48:57.861336 22337 leader_election.cc:288] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:48:57.861387 22342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:57.861555 22342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 LEADER]: Becoming Leader. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Running, Role: LEADER
I20260806 06:48:57.861637 22337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:48:57.861706 22342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:48:57.862126 22343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:48:57.862195 22343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:57.862421 22344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7616b60c69fd467a848c5d8fed919e3d. Latest consensus state: current_term: 1 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:48:57.862562 22344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:48:57.863827 22348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:48:57.864284 22348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:48:57.866046 22348 catalog_manager.cc:1347] Generated new cluster ID: 2d1cd699deb849b3a46e65e220d3117c
I20260806 06:48:57.866220 22348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:48:57 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:57.871178 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:57.858328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.6.57.126:41151)
0806 06:48:57.858561 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:57.858568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:57.858593 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:48:57.859542 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:57.859549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:57.859604 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:57.859707 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:57.860706 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:57.862437 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:57.863603 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:57.864214 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:57.864314 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:57.867289 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:57.867293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:57.867293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:57.867304 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:57.869350 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:57.869781 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:57.869785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:57.869786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:57.869941 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:57.870181 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:57.870183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:57.870183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:57.870550 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:57.870744 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:57.870828 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:48:57.870971 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260806 06:48:57.874946 22348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:48:57.875633 22348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:48:57.882248 22348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Generated new TSK 0
I20260806 06:48:57.882412 22348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:48:57.911475 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:57.993489 22364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:57.993681 22364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:57.993705 22364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:57.994799 22364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:57.994841 22364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:57.994867 22364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:48:57.996296 22364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:57.996474 22364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:57.996668 22364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:58.001559 22369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:58.001590 22370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:58.001734 22364 server_base.cc:1034] running on GCE node
W20260806 06:48:58.001573 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:58.002038 22364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:58.002287 22364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:58.003429 22364 hybrid_clock.cc:648] HybridClock initialized: now 1785998938003395 us; error 47 us; skew 500 ppm
Aug 06 06:48:57 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998937, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:58.005793 22364 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:48:58.006232 22364 webserver.cc:460] Webserver started at http://127.6.57.65:44601/ using document root <none> and password file <none>
I20260806 06:48:58.006403 22364 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:58.006443 22364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:58.006580 22364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:58.007503 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "d814d0e42586446a89552cbd512ee7b0"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "1cc0ab1dad6de8120209739238573066"
server_key_iv: "2ab0d1415db68d834a2a150a77305e73"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.007781 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "d814d0e42586446a89552cbd512ee7b0"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "1cc0ab1dad6de8120209739238573066"
server_key_iv: "2ab0d1415db68d834a2a150a77305e73"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.009021 22364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:48:58.009693 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.009912 22364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:48:58.009985 22364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "d814d0e42586446a89552cbd512ee7b0"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "1cc0ab1dad6de8120209739238573066"
server_key_iv: "2ab0d1415db68d834a2a150a77305e73"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.010036 22364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:58.023289 22364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:58.025398 22364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:58.025497 22364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:58.025655 22364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:58.025882 22364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:58.025910 22364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.025930 22364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:58.025942 22364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.031992 22364 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:42079
I20260806 06:48:58.032048 22499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:42079 every 8 connection(s)
I20260806 06:48:58.032368 22364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:48:58.036468 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22364
I20260806 06:48:58.036597 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:48:58.036810 6373 external_mini_cluster.cc:1428] Setting key 36ea81378747c238282359b8127d1a4c
Aug 06 06:48:58 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998937, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:58.041975 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.033008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:54539)
0806 06:48:58.033206 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.033239 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.033691 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.033938 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.033941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.034467 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:58.034653 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.034658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.035353 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.035354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.036134 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.036136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.036555 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.036556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.036705 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.037223 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:58.037231 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:58.038690 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:58.040610 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.040616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.040618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.040830 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.041066 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.041067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.041067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.041176 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.041486 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:58.041491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:58.041617 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.041702 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.041823 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260806 06:48:58.042029 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.033046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54539 (local address 127.6.57.126:41151)
0806 06:48:58.033115 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.033117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.033318 (+ 201us) server_negotiation.cc:407] Connection header received
0806 06:48:58.033796 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.033797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.033815 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.033870 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:58.034757 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.035254 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.036229 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.036455 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.036543 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.038787 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:58.038789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:58.038790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:58.038798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:58.040504 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.040905 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.040906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.040906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.040971 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.041253 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.041254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.041254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.041407 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:58.041488 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.041757 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.041881 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:48:58.042861 22500 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:48:58.042963 22500 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:58.043170 22500 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:48:58.043651 22299 ts_manager.cc:194] Registered new tserver with Master: d814d0e42586446a89552cbd512ee7b0 (127.6.57.65:42079)
I20260806 06:48:58.044283 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:54539
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:48:58.064680 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:58.145198 22505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:58.145447 22505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:58.145481 22505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:58.146570 22505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:58.146631 22505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:58.146672 22505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:48:58.148168 22505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:58.148418 22505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:58.148645 22505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:58.153981 22510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:58.154012 22513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:58.154055 22511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:58.154536 22505 server_base.cc:1034] running on GCE node
I20260806 06:48:58.154742 22505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:58.154966 22505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:58.156138 22505 hybrid_clock.cc:648] HybridClock initialized: now 1785998938156132 us; error 37 us; skew 500 ppm
Aug 06 06:48:58 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998938, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:58.158706 22505 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:48:58.159174 22505 webserver.cc:460] Webserver started at http://127.6.57.66:45513/ using document root <none> and password file <none>
I20260806 06:48:58.159391 22505 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:58.159435 22505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:58.159585 22505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:58.160576 22505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "9081fe93f01e40a3d99b7381d25a4628"
server_key_iv: "e3d2d1b94f8e8757707498ac02a8819a"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.160912 22505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "9081fe93f01e40a3d99b7381d25a4628"
server_key_iv: "e3d2d1b94f8e8757707498ac02a8819a"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.162319 22505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:48:58.163184 22520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.163373 22505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:48:58.163503 22505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "9081fe93f01e40a3d99b7381d25a4628"
server_key_iv: "e3d2d1b94f8e8757707498ac02a8819a"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.163621 22505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:58.177012 22505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:58.179756 22505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:58.179908 22505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:58.180112 22505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:58.180382 22505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:58.180420 22505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.180446 22505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:58.180464 22505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.187059 22505 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:35799
I20260806 06:48:58.187096 22638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:35799 every 8 connection(s)
I20260806 06:48:58.187525 22505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:48:58.189427 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22505
I20260806 06:48:58.189531 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:48:58.189714 6373 external_mini_cluster.cc:1428] Setting key baabd4b9da346a89f3b159abf8706c02
Aug 06 06:48:58 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998938, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:58.198817 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.188305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:34181 (local address 127.6.57.126:41151)
0806 06:48:58.188382 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.188385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.188599 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:48:58.189238 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.189239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.189254 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.189324 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:58.190449 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.191063 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.192446 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.192642 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.192709 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.195617 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:58.195619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:58.195619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:58.195627 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:58.197320 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.197701 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.197703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.197703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.197782 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.198101 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.198102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.198102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.198263 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:58.198341 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.198618 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.198714 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:48:58.198905 22641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.188271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:34181)
0806 06:48:58.188451 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.188503 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.189107 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.189377 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.189380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.190120 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:58.190331 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.190337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.191202 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.191203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.192341 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.192345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.192745 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.192747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.192943 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.193521 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:58.193533 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:58.195516 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:58.197402 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.197408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.197410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.197622 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.197852 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.197853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.197853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.198016 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.198336 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:58.198341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:58.198453 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.198587 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.198717 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260806 06:48:58.199506 22639 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:48:58.199613 22639 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:58.199803 22639 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:48:58.200232 22299 ts_manager.cc:194] Registered new tserver with Master: 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
I20260806 06:48:58.200596 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:34181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:48:58.217870 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:48:58.299705 22644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:48:58.299906 22644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:48:58.299929 22644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:48:58.301008 22644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:48:58.301040 22644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:48:58.301066 22644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:48:58.302560 22644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:48:58.302757 22644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:48:58.302922 22644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:48:58.307739 22652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:48:58.307729 22649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:58.307961 22644 server_base.cc:1034] running on GCE node
W20260806 06:48:58.307730 22650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:48:58.308245 22644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:48:58.308446 22644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:48:58.309582 22644 hybrid_clock.cc:648] HybridClock initialized: now 1785998938309572 us; error 30 us; skew 500 ppm
Aug 06 06:48:58 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998938, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:48:58.312026 22644 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:48:58.312520 22644 webserver.cc:460] Webserver started at http://127.6.57.67:43999/ using document root <none> and password file <none>
I20260806 06:48:58.312754 22644 fs_manager.cc:359] Metadata directory not provided
I20260806 06:48:58.312824 22644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:48:58.312975 22644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:48:58.313988 22644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "a5499c0719d1456db7c5c855364a3f17"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "a07fec599d28d43d467af399310fa97f"
server_key_iv: "7607cce0d111d3564e1ae2d7a5a2d49d"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.314349 22644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "a5499c0719d1456db7c5c855364a3f17"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "a07fec599d28d43d467af399310fa97f"
server_key_iv: "7607cce0d111d3564e1ae2d7a5a2d49d"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.315730 22644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:48:58.316439 22659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.316614 22644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:48:58.316713 22644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "a5499c0719d1456db7c5c855364a3f17"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "a07fec599d28d43d467af399310fa97f"
server_key_iv: "7607cce0d111d3564e1ae2d7a5a2d49d"
server_key_version: "encryptionkey@0"
I20260806 06:48:58.316794 22644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:48:58.338836 22644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:48:58.340891 22644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:48:58.341044 22644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:48:58.341275 22644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:48:58.341591 22644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:48:58.341653 22644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.341693 22644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:48:58.341720 22644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:48:58.347563 22644 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:43501
I20260806 06:48:58.347682 22775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:43501 every 8 connection(s)
I20260806 06:48:58.348008 22644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:48:58.353065 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22644
I20260806 06:48:58.353199 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:48:58.353411 6373 external_mini_cluster.cc:1428] Setting key 8a55c673b702fe176c50d9b31b258355
Aug 06 06:48:58 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998938, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:48:58.356943 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.348833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:34665 (local address 127.6.57.126:41151)
0806 06:48:58.348914 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.348916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.349077 (+ 161us) server_negotiation.cc:407] Connection header received
0806 06:48:58.349535 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.349536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.349547 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.349596 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:58.350546 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.351128 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.352091 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.352270 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.352322 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.354658 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:58.354660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:58.354660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:58.354666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:58.355799 (+ 1133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.356149 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.356150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.356150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.356201 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.356405 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.356406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.356406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.356515 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:58.356565 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.356816 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.356879 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 06:48:58.357185 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.348793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:34665)
0806 06:48:58.348957 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.348993 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.349440 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.349650 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.349653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.350218 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:48:58.350401 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.350406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.351236 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.351236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.352014 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.352016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.352343 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.352343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.352441 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.352891 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:48:58.352900 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:48:58.354567 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:48:58.355868 (+ 1301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.355874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.355875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.356081 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.356258 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:48:58.356259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:48:58.356259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:48:58.356351 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:48:58.356568 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:48:58.356572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:48:58.356684 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.356798 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.356959 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260806 06:48:58.357611 22776 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:48:58.357719 22776 heartbeater.cc:464] Registering TS with master...
I20260806 06:48:58.357901 22776 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:48:58.358261 22299 ts_manager.cc:194] Registered new tserver with Master: a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501)
I20260806 06:48:58.358678 22299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:34665
I20260806 06:48:58.365099 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:48:58.372192 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.366575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.6.57.126:41151)
0806 06:48:58.366622 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.366624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.366750 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:48:58.366788 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.366789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.366796 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.366845 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:48:58.367377 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.367887 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.368559 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.368734 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.368781 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.369844 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:48:58.369845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:48:58.369846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:48:58.369870 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:48:58.371286 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.371574 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.371574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.371575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.371631 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:48:58.371788 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:48:58.371788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:48:58.371788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:48:58.371904 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:48:58.371994 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.372075 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.372135 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:48:58.373592 22299 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:48:58.375228 22299 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:48:58.381639 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.6.57.66:35799)
0806 06:48:58.377663 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.377668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.377873 (+ 205us) server_negotiation.cc:407] Connection header received
0806 06:48:58.377955 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.377957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.377974 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.378061 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:58.378349 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379491 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.380771 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381233 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381319 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381376 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.381514 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.381552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260806 06:48:58.381664 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.0.0.1:58528)
0806 06:48:58.377671 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.377702 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.377787 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.378100 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.378103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.378108 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:58.378189 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.378190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379602 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.379602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.380691 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.380693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381320 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.381401 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381415 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.381487 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.381541 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":24,"thread_start_us":57,"threads_started":1}
I20260806 06:48:58.381827 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.6.57.67:43501)
0806 06:48:58.377802 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.377806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.377940 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:48:58.377978 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.377980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.377994 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.378310 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:58.378406 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379155 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.380548 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381231 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381300 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381353 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.381703 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.381751 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260806 06:48:58.381834 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:42079 (local address 127.0.0.1:56530)
0806 06:48:58.377644 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.377688 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.377771 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.378433 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.378434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.378436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:58.378516 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.378518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379721 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.379722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.380688 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.380690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381623 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.381686 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381689 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.381741 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.381773 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"spinlock_wait_cycles":384,"thread_start_us":54,"threads_started":1}
I20260806 06:48:58.381891 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43501 (local address 127.0.0.1:41280)
0806 06:48:58.377871 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.377880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.377920 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.378100 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.378103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.378105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:58.378188 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.378190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379258 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.379259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.380462 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.380465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381391 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.381466 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381472 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.381527 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.381568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260806 06:48:58.382247 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.377475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.6.57.65:42079)
0806 06:48:58.378209 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.378214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.378223 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:58.378261 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.378263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.378279 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.378351 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:58.378585 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.379618 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.380783 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.381481 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.381540 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.381589 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.382074 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.382137 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":106,"threads_started":1}
I20260806 06:48:58.382802 22707 tablet_service.cc:1467] Processing CreateTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 (DEFAULT_TABLE table=test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:58.382982 22570 tablet_service.cc:1467] Processing CreateTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 (DEFAULT_TABLE table=test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:58.383177 22707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6798510d3bc24cbf976d4d6e3332e5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:58.383351 22570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6798510d3bc24cbf976d4d6e3332e5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:58.385874 22790 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Bootstrap starting.
I20260806 06:48:58.385917 22791 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Bootstrap starting.
I20260806 06:48:58.386667 22791 tablet_bootstrap.cc:654] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:58.386684 22790 tablet_bootstrap.cc:654] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:58.386659 22429 tablet_service.cc:1467] Processing CreateTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 (DEFAULT_TABLE table=test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:48:58.387041 22429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6798510d3bc24cbf976d4d6e3332e5b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:48:58.387253 22791 log.cc:824] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:58.387283 22790 log.cc:824] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:58.388069 22790 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: No bootstrap required, opened a new log
I20260806 06:48:58.388139 22791 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: No bootstrap required, opened a new log
I20260806 06:48:58.388145 22790 ts_tablet_manager.cc:1397] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:48:58.388224 22791 ts_tablet_manager.cc:1397] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:48:58.389765 22791 raft_consensus.cc:348] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.389927 22791 raft_consensus.cc:374] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:58.389992 22791 raft_consensus.cc:729] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, State: Initialized, Role: FOLLOWER
I20260806 06:48:58.390163 22791 consensus_queue.cc:260] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.390218 22790 raft_consensus.cc:348] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.390396 22790 raft_consensus.cc:374] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:58.390450 22790 raft_consensus.cc:729] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5499c0719d1456db7c5c855364a3f17, State: Initialized, Role: FOLLOWER
I20260806 06:48:58.390465 22791 ts_tablet_manager.cc:1428] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:48:58.390627 22639 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
I20260806 06:48:58.390679 22790 consensus_queue.cc:260] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.390533 22794 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Bootstrap starting.
I20260806 06:48:58.390955 22790 ts_tablet_manager.cc:1428] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:48:58.390993 22776 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
I20260806 06:48:58.391732 22794 tablet_bootstrap.cc:654] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:48:58.392299 22794 log.cc:824] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:48:58.392935 22794 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: No bootstrap required, opened a new log
I20260806 06:48:58.392987 22794 ts_tablet_manager.cc:1397] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 06:48:58.394263 22794 raft_consensus.cc:348] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.394400 22794 raft_consensus.cc:374] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:48:58.394420 22794 raft_consensus.cc:729] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Initialized, Role: FOLLOWER
I20260806 06:48:58.394610 22794 consensus_queue.cc:260] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.394899 22794 ts_tablet_manager.cc:1428] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:48:58.394951 22500 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
I20260806 06:48:58.431164 22798 raft_consensus.cc:482] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:48:58.431269 22798 raft_consensus.cc:504] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.431628 22798 leader_election.cc:288] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799), a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501)
I20260806 06:48:58.434582 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.431785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.6.57.65:36209)
0806 06:48:58.431872 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.431885 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.432042 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.432253 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.432254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.432256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:58.432332 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.432334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.432945 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.432946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.433739 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.433741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.434289 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.434289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.434373 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.434376 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.434438 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.434479 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":64}
I20260806 06:48:58.434674 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.431876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54665 (local address 127.6.57.67:43501)
0806 06:48:58.431938 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.431940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.432106 (+ 166us) server_negotiation.cc:407] Connection header received
0806 06:48:58.432205 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.432207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.432234 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.432285 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:58.432461 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.432878 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.433831 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.434209 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.434289 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.434300 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.434559 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.434603 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:48:58.434690 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.431816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43501 (local address 127.6.57.65:54665)
0806 06:48:58.432023 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.432033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:48:58.432121 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:48:58.432303 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:48:58.432304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:48:58.432306 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:48:58.432395 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.432397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.432972 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.432973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.433740 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:48:58.433742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.434293 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.434294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:48:58.434367 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.434375 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 06:48:58.434551 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 06:48:58.434596 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":117,"threads_started":1}
I20260806 06:48:58.434885 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.431904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:36209 (local address 127.6.57.66:35799)
0806 06:48:58.432002 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.432004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.432014 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:48:58.432166 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.432168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.432180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.432240 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:48:58.432400 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.432870 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.433839 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.434208 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.434290 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.434300 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.434455 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.434825 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 06:48:58.435004 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6798510d3bc24cbf976d4d6e3332e5b1" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5499c0719d1456db7c5c855364a3f17" is_pre_election: true
I20260806 06:48:58.435163 22728 raft_consensus.cc:2454] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 0.
I20260806 06:48:58.435137 22590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6798510d3bc24cbf976d4d6e3332e5b1" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" is_pre_election: true
I20260806 06:48:58.435254 22590 raft_consensus.cc:2454] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 0.
I20260806 06:48:58.435387 22382 leader_election.cc:302] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5499c0719d1456db7c5c855364a3f17, d814d0e42586446a89552cbd512ee7b0; no voters:
I20260806 06:48:58.435506 22798 raft_consensus.cc:2781] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:48:58.435560 22798 raft_consensus.cc:482] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:48:58.435593 22798 raft_consensus.cc:3037] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:58.436287 22798 raft_consensus.cc:504] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.436425 22798 leader_election.cc:288] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 election: Requested vote from peers 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799), a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501)
I20260806 06:48:58.436576 22590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6798510d3bc24cbf976d4d6e3332e5b1" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
I20260806 06:48:58.436616 22728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6798510d3bc24cbf976d4d6e3332e5b1" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5499c0719d1456db7c5c855364a3f17"
I20260806 06:48:58.436662 22590 raft_consensus.cc:3037] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:58.436679 22728 raft_consensus.cc:3037] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:48:58.437538 22590 raft_consensus.cc:2454] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 1.
I20260806 06:48:58.437541 22728 raft_consensus.cc:2454] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 1.
I20260806 06:48:58.437744 22383 leader_election.cc:302] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, d814d0e42586446a89552cbd512ee7b0; no voters:
I20260806 06:48:58.437865 22798 raft_consensus.cc:2781] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:48:58.438046 22798 raft_consensus.cc:686] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 LEADER]: Becoming Leader. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Running, Role: LEADER
W20260806 06:48:58.438064 22640 tablet.cc:2369] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:48:58.438158 22798 consensus_queue.cc:237] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:48:58.439031 22299 catalog_manager.cc:5495] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d814d0e42586446a89552cbd512ee7b0 (127.6.57.65). New cstate: current_term: 1 leader_uuid: "d814d0e42586446a89552cbd512ee7b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } health_report { overall_health: UNKNOWN } } }
I20260806 06:48:58.453944 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.451796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56538 (local address 127.6.57.65:42079)
0806 06:48:58.451859 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 06:48:58.451860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:48:58.451869 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:48:58.451898 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:48:58.451898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:48:58.451928 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:48:58.452009 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:48:58.452130 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.452582 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.453293 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:48:58.453478 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:48:58.453541 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:48:58.453664 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:48:58.453749 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:48:58.453807 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:48:58.453834 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 06:48:58.453850 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:48:58.455605 22590 raft_consensus.cc:1261] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Refusing update from remote peer d814d0e42586446a89552cbd512ee7b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:58.455616 22728 raft_consensus.cc:1261] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Refusing update from remote peer d814d0e42586446a89552cbd512ee7b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:48:58.456004 22798 consensus_queue.cc:1035] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:58.456154 22800 consensus_queue.cc:1035] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:48:58.458787 22804 mvcc.cc:205] Tried to move back new op lower bound from 7315451651911659520 to 7315451651843141632. Current Snapshot: MvccSnapshot[applied={T|T < 7315451651911659520}]
I20260806 06:48:58.459445 22805 mvcc.cc:205] Tried to move back new op lower bound from 7315451651911659520 to 7315451651843141632. Current Snapshot: MvccSnapshot[applied={T|T < 7315451651911659520}]
I20260806 06:48:58.460762 22298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:58.460857 22298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:48:58.461705 22298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Sending DeleteTablet for 3 replicas of tablet 6798510d3bc24cbf976d4d6e3332e5b1
I20260806 06:48:58.462061 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22364
I20260806 06:48:58.462069 22570 tablet_service.cc:1514] Processing DeleteTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:58 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:58528
I20260806 06:48:58.464030 22707 tablet_service.cc:1514] Processing DeleteTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:58 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:41280
I20260806 06:48:58.464249 22813 tablet_replica.cc:330] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: stopping tablet replica
I20260806 06:48:58.464370 22813 raft_consensus.cc:2229] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:58.464478 22813 raft_consensus.cc:2258] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:58.464841 22813 ts_tablet_manager.cc:1905] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:48:58.466636 22813 ts_tablet_manager.cc:1918] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:48:58.466725 22813 log.cc:1196] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/6798510d3bc24cbf976d4d6e3332e5b1
I20260806 06:48:58.467013 22813 ts_tablet_manager.cc:1939] T 6798510d3bc24cbf976d4d6e3332e5b1 P a5499c0719d1456db7c5c855364a3f17: Deleting consensus metadata
I20260806 06:48:58.467406 22284 catalog_manager.cc:4841] TS a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): tablet 6798510d3bc24cbf976d4d6e3332e5b1 (table test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]) successfully deleted
W20260806 06:48:58.470258 22284 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:42079 (error 108)
I20260806 06:48:58.470577 22812 tablet_replica.cc:330] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: stopping tablet replica
I20260806 06:48:58.470579 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22505
I20260806 06:48:58.470681 22812 raft_consensus.cc:2229] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:48:58.470789 22812 raft_consensus.cc:2258] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:48:58.471465 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.470677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:42079 (local address 127.0.0.1:56552)
0806 06:48:58.471399 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.471439 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:42079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":700}
W20260806 06:48:58.471594 22284 catalog_manager.cc:4584] TS d814d0e42586446a89552cbd512ee7b0 (127.6.57.65:42079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6798510d3bc24cbf976d4d6e3332e5b1: Network error: Client connection negotiation failed: client connection to 127.6.57.65:42079: connect: Connection refused (error 111)
I20260806 06:48:58.477499 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22644
I20260806 06:48:58.477524 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:48:58.477428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.0.0.1:58544)
0806 06:48:58.477471 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:48:58.477499 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.66:35799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 06:48:58.483062 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22265
I20260806 06:49:18.489346 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--webserver_interface=127.6.57.126
--webserver_port=41447
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:18.570604 22815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:18.570789 22815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:18.570811 22815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:18.571841 22815 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:49:18.571872 22815 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:49:18.571885 22815 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:49:18.571899 22815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:18.571911 22815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:18.571923 22815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:18.571954 22815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:18.571969 22815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:18.573351 22815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=41447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:18.573525 22815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:18.573685 22815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:18.582337 22823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:18.582373 22822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:18.582413 22815 server_base.cc:1034] running on GCE node
W20260806 06:49:18.582374 22826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:18.582940 22815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:18.583339 22815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:18.584510 22815 hybrid_clock.cc:648] HybridClock initialized: now 1785998958584485 us; error 56 us; skew 500 ppm
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:18.592549 22815 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:49:18.593009 22815 webserver.cc:460] Webserver started at http://127.6.57.126:41447/ using document root <none> and password file <none>
I20260806 06:49:18.593262 22815 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:18.593353 22815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:18.594733 22815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:18.595449 22833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:18.595608 22815 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:18.595713 22815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "7616b60c69fd467a848c5d8fed919e3d"
format_stamp: "Formatted at 2026-08-06 06:48:57 on dist-test-slave-5l3k"
server_key: "d01edd62daab47905b0919bb987582f6"
server_key_iv: "4559bc5e75759678323fce81115d6ecb"
server_key_version: "encryptionkey@0"
I20260806 06:49:18.596127 22815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:18.627722 22815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:18.630757 22815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:18.630898 22815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:18.634976 22815 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41151
I20260806 06:49:18.635036 22888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41151 every 8 connection(s)
I20260806 06:49:18.635542 22815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:18.635803 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22815
I20260806 06:49:18.636162 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:42079
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=44601
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:49:18.637102 22889 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:49:18.637702 22889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap starting.
I20260806 06:49:18.648996 22889 log.cc:824] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:18.651592 22889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:49:18.651901 22889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap complete.
I20260806 06:49:18.655258 22889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:18.655588 22889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Initialized, Role: FOLLOWER
I20260806 06:49:18.655747 22889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:18.655809 22889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:18.655849 22889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:18.655892 22889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 1 FOLLOWER]: Advancing to term 2
I20260806 06:49:18.657135 22889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:18.657240 22889 leader_election.cc:302] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7616b60c69fd467a848c5d8fed919e3d; no voters:
I20260806 06:49:18.657392 22889 leader_election.cc:288] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 06:49:18.657471 22893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 06:49:18.657639 22889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:18.657670 22893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 LEADER]: Becoming Leader. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Running, Role: LEADER
I20260806 06:49:18.657931 22893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:18.658236 22894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:49:18.658316 22894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:18.658649 22895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7616b60c69fd467a848c5d8fed919e3d. Latest consensus state: current_term: 2 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:49:18.658706 22895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:18.658887 22902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:18.659673 22902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:18.659787 22902 catalog_manager.cc:1259] Loaded cluster ID: 2d1cd699deb849b3a46e65e220d3117c
I20260806 06:49:18.659818 22902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:18.660555 22902 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:18.660748 22902 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Loaded TSK: 0
I20260806 06:49:18.672757 22902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Generated new TSK 1
I20260806 06:49:18.672847 22902 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:49:18.726624 22891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:18.726840 22891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:18.726876 22891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:18.727965 22891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:18.728029 22891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:18.728070 22891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:18.729732 22891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:42079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=44601
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:18.730000 22891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:18.730233 22891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:18.736266 22919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:18.736279 22918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:18.736402 22922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:18.736500 22891 server_base.cc:1034] running on GCE node
I20260806 06:49:18.736713 22891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:18.736971 22891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:18.738202 22891 hybrid_clock.cc:648] HybridClock initialized: now 1785998958738175 us; error 43 us; skew 500 ppm
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:18.741092 22891 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:49:18.741605 22891 webserver.cc:460] Webserver started at http://127.6.57.65:44601/ using document root <none> and password file <none>
I20260806 06:49:18.741868 22891 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:18.741962 22891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:18.743440 22891 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:18.744175 22929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:18.744383 22891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:18.744494 22891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "d814d0e42586446a89552cbd512ee7b0"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "1cc0ab1dad6de8120209739238573066"
server_key_iv: "2ab0d1415db68d834a2a150a77305e73"
server_key_version: "encryptionkey@0"
I20260806 06:49:18.744819 22891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:18.758911 22891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:18.761098 22891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:18.761234 22891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:18.761473 22891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:18.761965 22936 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:49:18.763552 22891 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:49:18.763602 22891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:49:18.763636 22891 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:49:18.764307 22891 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:49:18.764353 22891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:18.764393 22936 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Bootstrap starting.
I20260806 06:49:18.772265 22891 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:42079
I20260806 06:49:18.772754 22891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:18.773177 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 22891
I20260806 06:49:18.773495 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:35799
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=45513
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:49:18.773366 23046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:42079 every 8 connection(s)
I20260806 06:49:18.776384 22936 log.cc:824] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:18.781365 22936 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:49:18.781737 22936 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Bootstrap complete.
I20260806 06:49:18.783090 22936 ts_tablet_manager.cc:1397] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.010s
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:18.784780 22936 raft_consensus.cc:348] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:49:18.785185 22936 raft_consensus.cc:729] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Initialized, Role: FOLLOWER
I20260806 06:49:18.785338 22936 consensus_queue.cc:260] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:49:18.785631 22936 ts_tablet_manager.cc:1428] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 06:49:18.789558 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.775626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:38129)
0806 06:49:18.776284 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:18.776333 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:18.777136 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:18.777778 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:18.777783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:18.778125 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:18.778378 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.778385 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.780156 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.780157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.781426 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.781430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.782393 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.782395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.782531 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.783131 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:18.783153 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:18.784982 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:18.787816 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.787823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.787825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.788096 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.788372 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.788373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.788374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.788509 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.788962 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:18.788968 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:18.789105 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 06:49:18.789202 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:49:18.789362 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":100,"threads_started":1}
I20260806 06:49:18.790350 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.775985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:38129 (local address 127.6.57.126:41151)
0806 06:49:18.776589 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:18.776598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:18.776651 (+ 53us) server_negotiation.cc:407] Connection header received
0806 06:49:18.777407 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:18.777414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:18.777471 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:18.777599 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:18.778556 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.780005 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.781578 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.782187 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.782287 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.785096 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:18.785100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:18.785100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:18.785112 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:18.787723 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.788185 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.788188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.788188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.788291 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.788576 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.788577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.788577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.788858 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:18.788933 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:18.789995 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0806 06:49:18.790148 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":88,"threads_started":1}
I20260806 06:49:18.791365 23047 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:49:18.791474 23047 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:18.791678 23047 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:49:18.792454 22851 ts_manager.cc:194] Registered new tserver with Master: d814d0e42586446a89552cbd512ee7b0 (127.6.57.65:42079)
I20260806 06:49:18.793677 22851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:38129
I20260806 06:49:18.794834 23047 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.65@KRBTEST.COM
I20260806 06:49:18.803022 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.793085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:42079 (local address 127.0.0.1:40584)
0806 06:49:18.793541 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:18.793575 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:18.793639 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:18.793956 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:18.793961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:18.794271 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:18.794324 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.794325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.795832 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.795833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.796749 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.796752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.797380 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.797381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.797454 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.797938 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:18.797943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:18.799719 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:18.801737 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.801744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.801749 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.801995 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.802233 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.802234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.802234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.802301 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.802668 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:18.802675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:18.802733 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 06:49:18.802817 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:49:18.802934 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":91,"threads_started":1}
I20260806 06:49:18.803057 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.793361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.6.57.65:42079)
0806 06:49:18.793797 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:18.793802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:18.793811 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:18.793843 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:18.793846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:18.793861 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:18.794401 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:18.794664 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.795726 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.796838 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.797294 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.797375 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.799807 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:18.799814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:18.799815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:18.799822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:18.801635 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.802089 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.802093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.802093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.802157 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.802374 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.802375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.802375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.802588 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:18.802672 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:18.802870 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 06:49:18.802955 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":78,"threads_started":1}
I20260806 06:49:18.803789 22967 tablet_service.cc:1514] Processing DeleteTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:58 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:40584
I20260806 06:49:18.804371 23057 tablet_replica.cc:330] stopping tablet replica
I20260806 06:49:18.804476 23057 raft_consensus.cc:2229] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:18.804541 23057 raft_consensus.cc:2258] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:18.804844 23057 ts_tablet_manager.cc:1905] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:18.807040 23057 ts_tablet_manager.cc:1918] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:18.807114 23057 log.cc:1196] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal/wals/6798510d3bc24cbf976d4d6e3332e5b1
I20260806 06:49:18.807330 23057 ts_tablet_manager.cc:1939] T 6798510d3bc24cbf976d4d6e3332e5b1 P d814d0e42586446a89552cbd512ee7b0: Deleting consensus metadata
I20260806 06:49:18.807983 22835 catalog_manager.cc:4841] TS d814d0e42586446a89552cbd512ee7b0 (127.6.57.65:42079): tablet 6798510d3bc24cbf976d4d6e3332e5b1 (table test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]) successfully deleted
W20260806 06:49:18.869930 23049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:18.870146 23049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:18.870173 23049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:18.871503 23049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:18.871554 23049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:18.871618 23049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:18.873318 23049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:35799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=45513
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:18.873593 23049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:18.873881 23049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:18.879761 23065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:18.879782 23062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:18.879762 23063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:18.879778 23049 server_base.cc:1034] running on GCE node
I20260806 06:49:18.880088 23049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:18.880275 23049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:18.881433 23049 hybrid_clock.cc:648] HybridClock initialized: now 1785998958881423 us; error 33 us; skew 500 ppm
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:18.884047 23049 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:49:18.884481 23049 webserver.cc:460] Webserver started at http://127.6.57.66:45513/ using document root <none> and password file <none>
I20260806 06:49:18.884737 23049 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:18.884816 23049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:18.886143 23049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:18.886876 23072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:18.887066 23049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:18.887161 23049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "9081fe93f01e40a3d99b7381d25a4628"
server_key_iv: "e3d2d1b94f8e8757707498ac02a8819a"
server_key_version: "encryptionkey@0"
I20260806 06:49:18.887470 23049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:18.903060 23049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:18.904882 23049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:18.905064 23049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:18.905292 23049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:18.905736 23079 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:49:18.907338 23049 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:49:18.907403 23049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:49:18.907444 23049 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:49:18.908113 23049 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:49:18.908169 23049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:18.908236 23079 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Bootstrap starting.
I20260806 06:49:18.914778 23049 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:35799
I20260806 06:49:18.914870 23189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:35799 every 8 connection(s)
I20260806 06:49:18.915206 23049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:49:18.918977 23079 log.cc:824] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:18.920046 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23049
I20260806 06:49:18.920374 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:43501
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=43999
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:49:18.921648 23079 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:49:18.921957 23079 tablet_bootstrap.cc:492] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Bootstrap complete.
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:18.923275 23079 ts_tablet_manager.cc:1397] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260806 06:49:18.924902 23079 raft_consensus.cc:348] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:49:18.925338 23079 raft_consensus.cc:729] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, State: Initialized, Role: FOLLOWER
I20260806 06:49:18.925501 23079 consensus_queue.cc:260] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } } peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } }
I20260806 06:49:18.925807 23079 ts_tablet_manager.cc:1428] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260806 06:49:18.926853 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.916153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:37921 (local address 127.6.57.126:41151)
0806 06:49:18.916221 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:18.916223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:18.916298 (+ 75us) server_negotiation.cc:407] Connection header received
0806 06:49:18.917103 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:18.917104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:18.917115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:18.917176 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:18.918154 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.918724 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.920259 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.920612 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.920693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.923643 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:18.923645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:18.923645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:18.923654 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:18.925198 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.925736 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.925737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.925738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.925803 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.926122 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.926123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.926124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.926243 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:18.926322 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:18.926667 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 06:49:18.926761 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:49:18.927021 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.915935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:37921)
0806 06:49:18.916147 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:18.916206 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:18.916973 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:18.917257 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:18.917262 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:18.917741 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:18.918048 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.918058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.918837 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.918838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.920113 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.920116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.920713 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.920715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.920877 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.921491 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:18.921501 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:18.923540 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:18.925294 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.925301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.925303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.925634 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.925933 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.925934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.925934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.926052 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.926314 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:18.926321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:18.926462 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:49:18.926638 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 06:49:18.926801 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260806 06:49:18.927513 23190 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:49:18.927623 23190 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:18.927830 23190 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:49:18.928463 22851 ts_manager.cc:194] Registered new tserver with Master: 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
I20260806 06:49:18.929080 22851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:37921
I20260806 06:49:18.930506 23190 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
Aug 06 06:49:18 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998958, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.66@KRBTEST.COM
I20260806 06:49:18.936161 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.928780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.0.0.1:33092)
0806 06:49:18.929223 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:18.929235 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:18.929296 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:18.929472 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:18.929473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:18.929642 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:18.929687 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.929689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.930441 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.930442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.931186 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:18.931188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.931711 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.931713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:18.931791 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.932154 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:18.932159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:18.933465 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:18.935076 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.935077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.935078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.935293 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.935493 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:18.935494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:18.935494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:18.935556 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:18.935832 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:18.935833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:18.935876 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 06:49:18.935963 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 06:49:18.936071 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427}
I20260806 06:49:18.936162 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:18.928836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.6.57.66:35799)
0806 06:49:18.929062 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:18.929067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:18.929289 (+ 222us) server_negotiation.cc:407] Connection header received
0806 06:49:18.929367 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:18.929368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:18.929380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:18.929462 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:18.929772 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.930361 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.931262 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:18.931620 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:18.931690 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:18.933548 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:18.933555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:18.933555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:18.933561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:18.934987 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.935358 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.935361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.935361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.935418 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:18.935624 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:18.935625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:18.935625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:18.935754 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:18.935839 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:18.935997 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 06:49:18.936080 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260806 06:49:18.936542 23122 tablet_service.cc:1514] Processing DeleteTablet for tablet 6798510d3bc24cbf976d4d6e3332e5b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:48:58 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:33092
I20260806 06:49:18.936990 23197 tablet_replica.cc:330] stopping tablet replica
I20260806 06:49:18.937093 23197 raft_consensus.cc:2229] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:18.937160 23197 raft_consensus.cc:2258] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:18.937428 23197 ts_tablet_manager.cc:1905] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:18.939901 23197 ts_tablet_manager.cc:1918] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:18.939976 23197 log.cc:1196] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/6798510d3bc24cbf976d4d6e3332e5b1
I20260806 06:49:18.940227 23197 ts_tablet_manager.cc:1939] T 6798510d3bc24cbf976d4d6e3332e5b1 P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting consensus metadata
I20260806 06:49:18.940716 22836 catalog_manager.cc:4841] TS 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): tablet 6798510d3bc24cbf976d4d6e3332e5b1 (table test-table [id=94b6ebb8bedd4013a04e6019d8b0202f]) successfully deleted
W20260806 06:49:19.008332 23194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:19.008513 23194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:19.008536 23194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:19.009639 23194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:19.009680 23194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:19.009727 23194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:19.011188 23194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:43501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=43999
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:19.011422 23194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:19.011677 23194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:19.016525 23203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:19.016546 23202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:19.016533 23205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:19.016816 23194 server_base.cc:1034] running on GCE node
I20260806 06:49:19.017045 23194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:19.017284 23194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:19.018461 23194 hybrid_clock.cc:648] HybridClock initialized: now 1785998959018447 us; error 42 us; skew 500 ppm
Aug 06 06:49:19 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998959, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:19.021003 23194 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:49:19.021390 23194 webserver.cc:460] Webserver started at http://127.6.57.67:43999/ using document root <none> and password file <none>
I20260806 06:49:19.021581 23194 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:19.021631 23194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:19.022993 23194 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 06:49:19.023581 23212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:19.023777 23194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:19.023852 23194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "a5499c0719d1456db7c5c855364a3f17"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "a07fec599d28d43d467af399310fa97f"
server_key_iv: "7607cce0d111d3564e1ae2d7a5a2d49d"
server_key_version: "encryptionkey@0"
I20260806 06:49:19.024184 23194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:19.044593 23194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:19.046311 23194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:19.046453 23194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:19.046685 23194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:19.046974 23194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:19.047006 23194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:19.047027 23194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:19.047073 23194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:19.052479 23194 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:43501
I20260806 06:49:19.052548 23330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:43501 every 8 connection(s)
I20260806 06:49:19.052888 23194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:49:19.056886 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23194
Aug 06 06:49:19 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998959, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:19.062484 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.053593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:59573 (local address 127.6.57.126:41151)
0806 06:49:19.053642 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.053645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.053859 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:49:19.054340 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.054342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.054354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.054401 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:19.055319 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.055826 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.056795 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.057041 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.057117 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.059651 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:19.059652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:19.059652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:19.059659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:19.061067 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.061542 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.061542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.061543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.061610 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.061839 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.061840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.061840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.061954 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:19.062057 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.062305 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.062391 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:49:19.062700 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.053587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:59573)
0806 06:49:19.053748 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:19.053779 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:19.054230 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:19.054466 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:19.054469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:19.055023 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:19.055210 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.055215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.055917 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.055918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.056659 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.056661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.057116 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.057117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.057272 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.057968 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:19.057976 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:19.059564 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:19.061147 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:19.061155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:19.061157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:19.061463 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:19.061672 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:19.061673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:19.061673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:19.061762 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:19.062040 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:19.062048 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:19.062173 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 06:49:19.062283 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 06:49:19.062429 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260806 06:49:19.063275 23331 heartbeater.cc:347] Connected to a master server at 127.6.57.126:41151
I20260806 06:49:19.063376 23331 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:19.063580 23331 heartbeater.cc:511] Master 127.6.57.126:41151 requested a full tablet report, sending...
I20260806 06:49:19.064004 22851 ts_manager.cc:194] Registered new tserver with Master: a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501)
I20260806 06:49:19.064508 22851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:59573
I20260806 06:49:19.064626 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.057297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.6.57.126:41151)
0806 06:49:19.057453 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.057456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.057500 (+ 44us) server_negotiation.cc:407] Connection header received
0806 06:49:19.057609 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.057610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.057621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.057671 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:19.058346 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.058996 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.059960 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.060152 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.060219 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.061546 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:19.061547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:19.061547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:19.061553 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:19.063462 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.063836 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.063837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.063838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.063896 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.064108 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.064109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.064109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.064228 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:19.064392 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.064426 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.064499 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
I20260806 06:49:19.065366 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:49:19.068279 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.065737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60890 (local address 127.6.57.126:41151)
0806 06:49:19.065786 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.065788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.065914 (+ 126us) server_negotiation.cc:407] Connection header received
0806 06:49:19.066013 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.066014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.066049 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.066146 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:19.066247 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.066839 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.067559 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.067762 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.067841 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.067999 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:19.068045 (+ 46us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 06:49:19.068222 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60890: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:49:19.068352 23335 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60890: token expired
W20260806 06:49:19.068442 22781 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.57.126:41151)
I20260806 06:49:19.074055 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.068687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.6.57.126:41151)
0806 06:49:19.068720 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.068722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.068744 (+ 22us) server_negotiation.cc:407] Connection header received
0806 06:49:19.068811 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.068811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.068820 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.068864 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:19.069323 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.069750 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.070430 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.070669 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.070749 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.071831 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:19.071833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:19.071833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:19.071839 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:19.072993 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.073321 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.073322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.073322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.073374 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:19.073580 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:19.073581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:19.073582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:19.073697 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:19.073792 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.073898 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.073968 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:49:19.075306 22851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:49:19.077036 22851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:49:19.079537 22967 tablet_service.cc:1467] Processing CreateTablet for tablet 9009d8728e16429a8ded30565831e8ee (DEFAULT_TABLE table=test-table [id=0ac66596d88f4c829432de6cb4d813b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:19.079511 23122 tablet_service.cc:1467] Processing CreateTablet for tablet 9009d8728e16429a8ded30565831e8ee (DEFAULT_TABLE table=test-table [id=0ac66596d88f4c829432de6cb4d813b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:19.079722 22967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9009d8728e16429a8ded30565831e8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:19.079723 23122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9009d8728e16429a8ded30565831e8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:19.081262 23079 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Bootstrap starting.
I20260806 06:49:19.081518 22936 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Bootstrap starting.
I20260806 06:49:19.081570 23079 tablet_bootstrap.cc:654] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:19.081845 22936 tablet_bootstrap.cc:654] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:19.082415 23079 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: No bootstrap required, opened a new log
I20260806 06:49:19.082428 22936 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: No bootstrap required, opened a new log
I20260806 06:49:19.082469 22936 ts_tablet_manager.cc:1397] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:19.082468 23079 ts_tablet_manager.cc:1397] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:49:19.082656 23079 raft_consensus.cc:348] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.082734 23079 raft_consensus.cc:374] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:19.082765 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.079253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43501 (local address 127.0.0.1:55124)
0806 06:49:19.079305 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:19.079316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:19.079355 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:19.079706 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:19.079707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:19.079709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:19.079746 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.079747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.080726 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.080727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.081580 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.081583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.082515 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.082516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.082579 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.082599 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 06:49:19.082660 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:49:19.082701 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 06:49:19.082783 23079 raft_consensus.cc:729] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, State: Initialized, Role: FOLLOWER
I20260806 06:49:19.082785 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.079354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.6.57.67:43501)
0806 06:49:19.079507 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.079511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.079520 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:19.079553 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.079560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.079574 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.079819 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:19.079902 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.080630 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.081663 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.082407 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.082518 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.082587 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.082675 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.082711 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260806 06:49:19.082830 23079 consensus_queue.cc:260] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.082850 22936 raft_consensus.cc:348] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.082923 23079 ts_tablet_manager.cc:1428] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:19.082913 22936 raft_consensus.cc:374] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:19.082955 22936 raft_consensus.cc:729] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Initialized, Role: FOLLOWER
I20260806 06:49:19.083003 22936 consensus_queue.cc:260] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.083106 22936 ts_tablet_manager.cc:1428] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 06:49:19.083779 23262 tablet_service.cc:1467] Processing CreateTablet for tablet 9009d8728e16429a8ded30565831e8ee (DEFAULT_TABLE table=test-table [id=0ac66596d88f4c829432de6cb4d813b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:19.084120 23262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9009d8728e16429a8ded30565831e8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:19.086830 23340 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Bootstrap starting.
I20260806 06:49:19.087440 23340 tablet_bootstrap.cc:654] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:19.087957 23340 log.cc:824] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:19.088718 23340 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: No bootstrap required, opened a new log
I20260806 06:49:19.088793 23340 ts_tablet_manager.cc:1397] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:19.090245 23340 raft_consensus.cc:348] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.090373 23340 raft_consensus.cc:374] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:19.090394 23340 raft_consensus.cc:729] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5499c0719d1456db7c5c855364a3f17, State: Initialized, Role: FOLLOWER
I20260806 06:49:19.090513 23340 consensus_queue.cc:260] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.090788 23340 ts_tablet_manager.cc:1428] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:19.090817 23331 heartbeater.cc:503] Master 127.6.57.126:41151 was elected leader, sending a full tablet report...
I20260806 06:49:19.115005 23054 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:19.115142 23054 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.115442 23054 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
I20260806 06:49:19.118800 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.115644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:44175 (local address 127.6.57.67:43501)
0806 06:49:19.115747 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.115749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.115757 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:49:19.115781 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.115781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.115792 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.115871 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:19.116042 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.116655 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.117872 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.118325 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.118381 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.118390 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.118669 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.118730 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 06:49:19.118800 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.115552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43501 (local address 127.6.57.65:44175)
0806 06:49:19.115590 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:19.115602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:19.115657 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:19.115879 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:19.115880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:19.115882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:19.115974 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.115977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.116756 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.116757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.117765 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.117768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.118421 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.118422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.118553 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.118580 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 06:49:19.118662 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:49:19.118722 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:49:19.119079 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.115678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:42347 (local address 127.6.57.66:35799)
0806 06:49:19.115745 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.115747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.115881 (+ 134us) server_negotiation.cc:407] Connection header received
0806 06:49:19.115908 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.115909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.115932 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.115989 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:19.116239 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.116945 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.118070 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.118661 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.118740 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.118823 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.118976 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.119018 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:49:19.119076 23343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.115599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.6.57.65:42347)
0806 06:49:19.115776 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:19.115785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:19.115861 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:19.116035 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:19.116036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:19.116038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:19.116115 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.116117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.117110 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.117111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.117933 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:19.117935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.118752 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.118753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:19.118858 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.118863 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:49:19.118963 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:49:19.119002 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260806 06:49:19.119203 23282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9009d8728e16429a8ded30565831e8ee" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5499c0719d1456db7c5c855364a3f17" is_pre_election: true
I20260806 06:49:19.119392 23282 raft_consensus.cc:2454] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 0.
I20260806 06:49:19.119356 23143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9009d8728e16429a8ded30565831e8ee" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" is_pre_election: true
I20260806 06:49:19.119508 23143 raft_consensus.cc:2454] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 0.
I20260806 06:49:19.119727 22931 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5499c0719d1456db7c5c855364a3f17, d814d0e42586446a89552cbd512ee7b0; no voters:
I20260806 06:49:19.119902 23054 raft_consensus.cc:2781] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:49:19.119971 23054 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:49:19.120018 23054 raft_consensus.cc:3037] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:19.120774 23054 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.120940 23054 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 election: Requested vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
I20260806 06:49:19.121124 23143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9009d8728e16429a8ded30565831e8ee" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
I20260806 06:49:19.121208 23143 raft_consensus.cc:3037] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:19.121166 23282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9009d8728e16429a8ded30565831e8ee" candidate_uuid: "d814d0e42586446a89552cbd512ee7b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5499c0719d1456db7c5c855364a3f17"
I20260806 06:49:19.121275 23282 raft_consensus.cc:3037] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:19.121857 23143 raft_consensus.cc:2454] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 1.
I20260806 06:49:19.122063 22932 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, d814d0e42586446a89552cbd512ee7b0; no voters:
I20260806 06:49:19.122141 23282 raft_consensus.cc:2454] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d814d0e42586446a89552cbd512ee7b0 in term 1.
I20260806 06:49:19.122225 23054 raft_consensus.cc:2781] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:19.122437 23054 raft_consensus.cc:686] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 LEADER]: Becoming Leader. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Running, Role: LEADER
I20260806 06:49:19.122565 23054 consensus_queue.cc:237] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:19.123476 22849 catalog_manager.cc:5495] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d814d0e42586446a89552cbd512ee7b0 (127.6.57.65). New cstate: current_term: 1 leader_uuid: "d814d0e42586446a89552cbd512ee7b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } health_report { overall_health: HEALTHY } } }
I20260806 06:49:19.129781 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.127603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.6.57.65:42079)
0806 06:49:19.127671 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:19.127673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:19.127683 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:49:19.127740 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:19.127741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:19.127755 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:19.127806 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:19.127935 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.128390 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.129014 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:19.129221 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:19.129277 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:19.129517 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:19.129619 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:49:19.129677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:19.129717 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:49:19.129736 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:49:19.131125 23282 raft_consensus.cc:1261] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Refusing update from remote peer d814d0e42586446a89552cbd512ee7b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:19.131125 23143 raft_consensus.cc:1261] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Refusing update from remote peer d814d0e42586446a89552cbd512ee7b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:19.131469 23344 consensus_queue.cc:1035] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:19.131575 23054 consensus_queue.cc:1035] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:19.134002 23347 mvcc.cc:205] Tried to move back new op lower bound from 7315451736598806528 to 7315451736566902784. Current Snapshot: MvccSnapshot[applied={T|T < 7315451736598806528}]
I20260806 06:49:19.134217 23348 mvcc.cc:205] Tried to move back new op lower bound from 7315451736598806528 to 7315451736566902784. Current Snapshot: MvccSnapshot[applied={T|T < 7315451736598806528}]
I20260806 06:49:19.135764 22849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:19.135859 22849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:19.136668 22849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Sending DeleteTablet for 3 replicas of tablet 9009d8728e16429a8ded30565831e8ee
I20260806 06:49:19.137010 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22891
I20260806 06:49:19.137013 23122 tablet_service.cc:1514] Processing DeleteTablet for tablet 9009d8728e16429a8ded30565831e8ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:19 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:33092
I20260806 06:49:19.137110 22967 tablet_service.cc:1514] Processing DeleteTablet for tablet 9009d8728e16429a8ded30565831e8ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:19 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:40584
I20260806 06:49:19.137100 23262 tablet_service.cc:1514] Processing DeleteTablet for tablet 9009d8728e16429a8ded30565831e8ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:19 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:55124
I20260806 06:49:19.137813 23197 tablet_replica.cc:330] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: stopping tablet replica
I20260806 06:49:19.137869 23197 raft_consensus.cc:2229] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:19.137928 23197 raft_consensus.cc:2258] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:19.138616 23355 tablet_replica.cc:330] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: stopping tablet replica
I20260806 06:49:19.138710 23355 raft_consensus.cc:2229] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:19.138767 23197 ts_tablet_manager.cc:1905] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:19.138798 23355 raft_consensus.cc:2258] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:19.139210 23355 ts_tablet_manager.cc:1905] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:19.140350 23355 ts_tablet_manager.cc:1918] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:19.140410 23355 log.cc:1196] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/9009d8728e16429a8ded30565831e8ee
I20260806 06:49:19.140511 23197 ts_tablet_manager.cc:1918] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:19.140564 23197 log.cc:1196] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/9009d8728e16429a8ded30565831e8ee
I20260806 06:49:19.140614 23355 ts_tablet_manager.cc:1939] T 9009d8728e16429a8ded30565831e8ee P a5499c0719d1456db7c5c855364a3f17: Deleting consensus metadata
I20260806 06:49:19.140784 23197 ts_tablet_manager.cc:1939] T 9009d8728e16429a8ded30565831e8ee P 80ec2bbbc15d4db9b3b83d85b8eb3eaa: Deleting consensus metadata
I20260806 06:49:19.141080 22835 catalog_manager.cc:4841] TS a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): tablet 9009d8728e16429a8ded30565831e8ee (table test-table [id=0ac66596d88f4c829432de6cb4d813b0]) successfully deleted
I20260806 06:49:19.141332 22836 catalog_manager.cc:4841] TS 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): tablet 9009d8728e16429a8ded30565831e8ee (table test-table [id=0ac66596d88f4c829432de6cb4d813b0]) successfully deleted
W20260806 06:49:19.145253 22835 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:42079 (error 108)
I20260806 06:49:19.145519 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23049
I20260806 06:49:19.145740 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:19.145614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:42079 (local address 127.0.0.1:40604)
0806 06:49:19.145669 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:19.145704 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:42079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 06:49:19.147235 22835 catalog_manager.cc:4584] TS d814d0e42586446a89552cbd512ee7b0 (127.6.57.65:42079): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9009d8728e16429a8ded30565831e8ee: Network error: Client connection negotiation failed: client connection to 127.6.57.65:42079: connect: Connection refused (error 111)
I20260806 06:49:19.151105 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23194
I20260806 06:49:19.156167 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 22815
I20260806 06:49:39.162358 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--webserver_interface=127.6.57.126
--webserver_port=41447
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:39.245566 23357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:39.245777 23357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:39.245803 23357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:39.247056 23357 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 06:49:39.247100 23357 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 06:49:39.247116 23357 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 06:49:39.247131 23357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:39.247148 23357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:39.247210 23357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:39.247241 23357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:39.247272 23357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:39.249003 23357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=41447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:39.249279 23357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:39.249564 23357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:39.255139 23362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.255227 23363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.255198 23365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.255208 23357 server_base.cc:1034] running on GCE node
I20260806 06:49:39.255663 23357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:39.255950 23357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:39.257139 23357 hybrid_clock.cc:648] HybridClock initialized: now 1785998979257122 us; error 26 us; skew 500 ppm
Aug 06 06:49:39 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:39.259861 23357 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:49:39.260404 23357 webserver.cc:460] Webserver started at http://127.6.57.126:41447/ using document root <none> and password file <none>
I20260806 06:49:39.260653 23357 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:39.260723 23357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:39.262140 23357 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:39.262892 23372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.263079 23357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.263182 23357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "7616b60c69fd467a848c5d8fed919e3d"
format_stamp: "Formatted at 2026-08-06 06:48:57 on dist-test-slave-5l3k"
server_key: "d01edd62daab47905b0919bb987582f6"
server_key_iv: "4559bc5e75759678323fce81115d6ecb"
server_key_version: "encryptionkey@0"
I20260806 06:49:39.263516 23357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:39.271579 23357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:39.273262 23357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:39.273420 23357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:39.277756 23357 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41151
I20260806 06:49:39.277844 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41151 every 8 connection(s)
I20260806 06:49:39.278215 23357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:39.279721 23426 sys_catalog.cc:263] Verifying existing consensus state
I20260806 06:49:39.280385 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap starting.
I20260806 06:49:39.288508 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23357
I20260806 06:49:39.288841 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:42079
--local_ip_for_outbound_sockets=127.6.57.65
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=44601
--webserver_interface=127.6.57.65
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:49:39.289117 23426 log.cc:824] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:39.291672 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:49:39.291985 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Bootstrap complete.
I20260806 06:49:39.294196 23426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:39.294412 23426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Initialized, Role: FOLLOWER
I20260806 06:49:39.294602 23426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:39.294668 23426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:39.294696 23426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:39.294763 23426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 2 FOLLOWER]: Advancing to term 3
I20260806 06:49:39.295737 23426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:39.295820 23426 leader_election.cc:302] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7616b60c69fd467a848c5d8fed919e3d; no voters:
I20260806 06:49:39.296010 23426 leader_election.cc:288] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 06:49:39.296052 23431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 06:49:39.296219 23431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [term 3 LEADER]: Becoming Leader. State: Replica: 7616b60c69fd467a848c5d8fed919e3d, State: Running, Role: LEADER
I20260806 06:49:39.296286 23426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:39.296352 23431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } }
I20260806 06:49:39.296653 23431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7616b60c69fd467a848c5d8fed919e3d. Latest consensus state: current_term: 3 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:49:39.296752 23431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:39.296658 23432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7616b60c69fd467a848c5d8fed919e3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7616b60c69fd467a848c5d8fed919e3d" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41151 } } }
I20260806 06:49:39.296927 23432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:39.297080 23439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:39.298007 23439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:39.298149 23439 catalog_manager.cc:1259] Loaded cluster ID: 2d1cd699deb849b3a46e65e220d3117c
I20260806 06:49:39.298427 23439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:39.299718 23439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:39.299897 23439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Loaded TSK: 0
I20260806 06:49:39.299949 23439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7616b60c69fd467a848c5d8fed919e3d: Loaded TSK: 1
F20260806 06:49:39.300164 23439 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4b8f6fe9ad google::LogMessage::Fail() at ??:0
@ 0x7f4b8f70188a google::LogMessage::SendToLog() at ??:0
@ 0x7f4b8f6fe4de google::LogMessage::Flush() at ??:0
@ 0x7f4b8f6ffa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4b91e6b104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4b91e68d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4b976581cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4b9765848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4b97658594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4b9761ec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4b97628ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4b915cc0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4b915c1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4b924256db start_thread at ??:0
@ 0x7f4b8e1fc61f clone at ??:0
W20260806 06:49:39.390362 23429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:39.390604 23429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:39.390630 23429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:39.391772 23429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:39.391810 23429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:39.391835 23429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:39.393204 23429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:42079
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=44601
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:39.393424 23429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:39.393651 23429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:39.398455 23471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.398547 23470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.398603 23429 server_base.cc:1034] running on GCE node
W20260806 06:49:39.398715 23474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.398903 23429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:39.399169 23429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:39.400336 23429 hybrid_clock.cc:648] HybridClock initialized: now 1785998979400325 us; error 27 us; skew 500 ppm
Aug 06 06:49:39 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:39.402973 23429 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:49:39.403438 23429 webserver.cc:460] Webserver started at http://127.6.57.65:44601/ using document root <none> and password file <none>
I20260806 06:49:39.403643 23429 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:39.403703 23429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:39.405086 23429 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:39.405819 23481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.405988 23429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:39.406118 23429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "d814d0e42586446a89552cbd512ee7b0"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "1cc0ab1dad6de8120209739238573066"
server_key_iv: "2ab0d1415db68d834a2a150a77305e73"
server_key_version: "encryptionkey@0"
I20260806 06:49:39.406452 23429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:39.430223 23429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:39.432435 23429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:39.432603 23429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:39.432840 23429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:39.433298 23488 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 06:49:39.435418 23429 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 06:49:39.435484 23429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 06:49:39.435519 23429 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 06:49:39.436187 23429 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 06:49:39.436239 23429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:39.436331 23488 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Bootstrap starting.
I20260806 06:49:39.443349 23429 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:42079
I20260806 06:49:39.443414 23598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:42079 every 8 connection(s)
I20260806 06:49:39.443781 23429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:39.445077 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23429
I20260806 06:49:39.445394 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:35799
--local_ip_for_outbound_sockets=127.6.57.66
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=45513
--webserver_interface=127.6.57.66
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:49:39.447368 23488 log.cc:824] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:39.450289 23488 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 06:49:39.450673 23488 tablet_bootstrap.cc:492] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Bootstrap complete.
I20260806 06:49:39.451799 23488 ts_tablet_manager.cc:1397] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260806 06:49:39.453266 23488 raft_consensus.cc:348] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:39.453599 23488 raft_consensus.cc:729] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d814d0e42586446a89552cbd512ee7b0, State: Initialized, Role: FOLLOWER
I20260806 06:49:39.453763 23488 consensus_queue.cc:260] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:39.454093 23488 ts_tablet_manager.cc:1428] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260806 06:49:39.494647 23601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:49:39.444567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:53321)
0806 06:49:39.444750 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.444795 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:39.445460 (+ 665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:39.494387 (+ 48927us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
W20260806 06:49:39.494915 23599 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:41151 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:41151: Client connection negotiation failed: client connection to 127.6.57.126:41151: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:49:39.495297 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.495179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:34691)
0806 06:49:39.495237 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.495278 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:49:39.495677 23601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.495599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:56533)
0806 06:49:39.495639 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.495662 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:49:39.495797 23599 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 06:49:39.533072 23602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:39.533281 23602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:39.533304 23602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:39.534442 23602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:39.534509 23602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:39.534538 23602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:39.535943 23602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:35799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=45513
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:39.536162 23602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:39.536381 23602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:39.541740 23612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.541810 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.541826 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.541855 23602 server_base.cc:1034] running on GCE node
I20260806 06:49:39.542223 23602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:39.542527 23602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:39.543684 23602 hybrid_clock.cc:648] HybridClock initialized: now 1785998979543673 us; error 30 us; skew 500 ppm
Aug 06 06:49:39 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:39.546182 23602 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:49:39.546677 23602 webserver.cc:460] Webserver started at http://127.6.57.66:45513/ using document root <none> and password file <none>
I20260806 06:49:39.546891 23602 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:39.546958 23602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:39.548261 23602 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:39.548873 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.549059 23602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:39.549166 23602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "9081fe93f01e40a3d99b7381d25a4628"
server_key_iv: "e3d2d1b94f8e8757707498ac02a8819a"
server_key_version: "encryptionkey@0"
I20260806 06:49:39.549503 23602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:39.569226 23602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:39.571350 23602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:39.571575 23602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:39.571825 23602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:39.572184 23602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:39.572245 23602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.572291 23602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:39.572326 23602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.578896 23602 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:35799
I20260806 06:49:39.578955 23737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:35799 every 8 connection(s)
I20260806 06:49:39.579309 23602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:49:39.580312 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.579904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:50339)
0806 06:49:39.580101 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.580172 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
W20260806 06:49:39.580713 23738 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:41151 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:41151: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
I20260806 06:49:39.581039 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.580909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:43743)
0806 06:49:39.580960 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.581020 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:49:39.581362 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.581255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:47755)
0806 06:49:39.581316 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.581338 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 06:49:39.581460 23738 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:49:39.583804 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23602
I20260806 06:49:39.584110 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:43501
--local_ip_for_outbound_sockets=127.6.57.67
--tserver_master_addrs=127.6.57.126:41151
--webserver_port=43999
--webserver_interface=127.6.57.67
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:40873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:39.664245 23741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:39.664431 23741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:39.664458 23741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:39.665597 23741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:39.665656 23741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:39.665704 23741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:39.667237 23741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:40873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:43501
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=43999
--tserver_master_addrs=127.6.57.126:41151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:39.667443 23741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:39.667659 23741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:39.672433 23749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:39.672441 23746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.672626 23741 server_base.cc:1034] running on GCE node
W20260806 06:49:39.672696 23747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:39.672895 23741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:39.673156 23741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:39.674319 23741 hybrid_clock.cc:648] HybridClock initialized: now 1785998979674321 us; error 41 us; skew 500 ppm
Aug 06 06:49:39 dist-test-slave-5l3k krb5kdc[22252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:39.676728 23741 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:49:39.677150 23741 webserver.cc:460] Webserver started at http://127.6.57.67:43999/ using document root <none> and password file <none>
I20260806 06:49:39.677392 23741 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:39.677459 23741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:39.678946 23741 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:49:39.679571 23756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.679772 23741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:39.679872 23741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "a5499c0719d1456db7c5c855364a3f17"
format_stamp: "Formatted at 2026-08-06 06:48:58 on dist-test-slave-5l3k"
server_key: "a07fec599d28d43d467af399310fa97f"
server_key_iv: "7607cce0d111d3564e1ae2d7a5a2d49d"
server_key_version: "encryptionkey@0"
I20260806 06:49:39.680204 23741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:39.688498 23741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:39.690208 23741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:39.690367 23741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:39.690645 23741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:39.690950 23741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:39.691004 23741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.691035 23741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:39.691061 23741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:39.696704 23741 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:43501
I20260806 06:49:39.696774 23873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:43501 every 8 connection(s)
I20260806 06:49:39.697113 23741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:49:39.698113 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.697752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:46649)
0806 06:49:39.697914 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.697977 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
W20260806 06:49:39.698519 23874 heartbeater.cc:644] Failed to heartbeat to 127.6.57.126:41151 (0 consecutive failures): Network error: Failed to ping master at 127.6.57.126:41151: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
I20260806 06:49:39.698772 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.698686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:40083)
0806 06:49:39.698728 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.698754 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:49:39.699030 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:39.698963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:46027)
0806 06:49:39.698996 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:39.699016 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:49:39.699142 23874 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 06:49:39.699600 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23741
I20260806 06:49:40.496403 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.496149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:34005)
0806 06:49:40.496292 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:40.496342 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260806 06:49:40.581929 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.581743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:52367)
0806 06:49:40.581849 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:40.581880 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
I20260806 06:49:40.694260 23880 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:40.694391 23880 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:40.694857 23880 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
I20260806 06:49:40.699653 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.699457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:33973)
0806 06:49:40.699577 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:40.699617 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
Aug 06 06:49:40 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.66@KRBTEST.COM
Aug 06 06:49:40 dist-test-slave-5l3k krb5kdc[22252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998979, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.67@KRBTEST.COM
I20260806 06:49:40.706121 23882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.695103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:49783 (local address 127.6.57.67:43501)
0806 06:49:40.695300 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:40.695310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:40.695353 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:49:40.695797 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:40.695800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:40.695847 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:40.696063 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:40.696812 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.698119 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.699572 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.700116 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.700192 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:40.702466 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:40.702470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:40.702508 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:40.702525 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:40.704291 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704685 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:40.704689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:40.704689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:40.704800 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:40.705121 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:40.705121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:40.705122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:40.705448 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:40.705547 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:40.705887 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 06:49:40.706018 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":59,"threads_started":1}
I20260806 06:49:40.706128 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.695149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:45755 (local address 127.6.57.66:35799)
0806 06:49:40.695318 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:40.695328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:40.695357 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:49:40.695796 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:40.695798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:40.695837 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:40.696026 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:40.696790 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.697650 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.699085 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.699636 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.699744 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:40.702117 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:40.702121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:40.702121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:40.702131 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:40.704099 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704686 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:40.704689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:40.704689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:40.704800 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:40.705121 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:40.705122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:40.705122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:40.705448 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:40.705538 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:40.705887 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 06:49:40.706028 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":51,"threads_started":1}
I20260806 06:49:40.706166 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.695044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:35799 (local address 127.6.57.65:45755)
0806 06:49:40.695160 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:40.695168 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:40.695208 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:40.696048 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:40.696051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:40.696439 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:40.696649 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:40.696657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.697799 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.697800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:40.698968 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:40.698981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.699735 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.699737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:40.699862 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:40.700486 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:40.700495 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:40.701981 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:40.704185 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:40.704193 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:40.704584 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:40.704883 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704883 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:40.704884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:40.705042 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:40.705535 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:40.705543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:40.705695 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:49:40.705812 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:49:40.706030 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":6,"thread_start_us":50,"threads_started":1}
I20260806 06:49:40.706219 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:40.694980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:43501 (local address 127.6.57.65:49783)
0806 06:49:40.695107 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:40.695120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:40.695187 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:40.696050 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:40.696051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:40.696438 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:40.696649 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:40.696657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.698201 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.698202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:40.698967 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:40.698981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:40.700195 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:40.700196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:40.700253 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:40.700659 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:40.700664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:40.702366 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:40.704365 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:40.704366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:40.704605 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:40.704883 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:40.704883 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:40.704884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:40.705042 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:40.705535 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:40.705543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:40.705695 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 06:49:40.705812 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 06:49:40.706086 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":151}
W20260806 06:49:40.707012 23484 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
W20260806 06:49:40.707176 23483 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
I20260806 06:49:40.707216 23483 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d814d0e42586446a89552cbd512ee7b0; no voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, a5499c0719d1456db7c5c855364a3f17
I20260806 06:49:40.707306 23880 raft_consensus.cc:2726] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:49:41.497257 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:41.497027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:34627)
0806 06:49:41.497179 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:41.497212 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260806 06:49:41.582784 23886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:41.582568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:33155)
0806 06:49:41.582714 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:41.582747 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260806 06:49:41.700390 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:41.700196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:34421)
0806 06:49:41.700319 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:41.700351 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260806 06:49:42.498062 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:42.497846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:58221)
0806 06:49:42.497981 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:42.498016 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260806 06:49:42.541621 23889 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:42.541742 23889 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:42.542017 23889 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
W20260806 06:49:42.542505 23483 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
W20260806 06:49:42.542591 23484 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
I20260806 06:49:42.542680 23484 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d814d0e42586446a89552cbd512ee7b0; no voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, a5499c0719d1456db7c5c855364a3f17
I20260806 06:49:42.542757 23889 raft_consensus.cc:2726] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:49:42.583449 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:42.583218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:57823)
0806 06:49:42.583355 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:42.583389 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260806 06:49:42.701102 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:42.700886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:42429)
0806 06:49:42.701011 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:42.701052 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260806 06:49:43.498852 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:43.498593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:41609)
0806 06:49:43.498737 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:43.498791 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260806 06:49:43.584251 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:43.584018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:50353)
0806 06:49:43.584163 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:43.584206 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260806 06:49:43.701759 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:43.701550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:37065)
0806 06:49:43.701682 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:43.701718 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":57,"threads_started":1}
I20260806 06:49:44.499614 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:44.499378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:45377)
0806 06:49:44.499532 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:44.499567 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260806 06:49:44.584903 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:44.584693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:47119)
0806 06:49:44.584826 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:44.584862 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260806 06:49:44.702369 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:44.702162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:49161)
0806 06:49:44.702296 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:44.702329 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260806 06:49:44.774621 23898 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:44.774739 23898 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:44.774904 23898 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
W20260806 06:49:44.775323 23483 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
W20260806 06:49:44.775411 23484 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
I20260806 06:49:44.775503 23484 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d814d0e42586446a89552cbd512ee7b0; no voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, a5499c0719d1456db7c5c855364a3f17
I20260806 06:49:44.775597 23898 raft_consensus.cc:2726] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:49:45.500231 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:45.500029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:55465)
0806 06:49:45.500152 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:45.500183 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 06:49:45.585546 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:45.585344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:50167)
0806 06:49:45.585470 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:45.585504 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260806 06:49:45.703047 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:45.702791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:53571)
0806 06:49:45.702968 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:45.703004 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260806 06:49:46.500945 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:46.500683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:48255)
0806 06:49:46.500868 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:46.500899 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":124,"threads_started":1}
I20260806 06:49:46.586174 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:46.585951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:49221)
0806 06:49:46.586104 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:46.586136 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260806 06:49:46.703727 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:46.703498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:47195)
0806 06:49:46.703655 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:46.703687 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260806 06:49:47.501703 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:47.501432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:53395)
0806 06:49:47.501619 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:47.501656 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260806 06:49:47.586886 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:47.586673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:54121)
0806 06:49:47.586808 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:47.586845 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260806 06:49:47.704367 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:47.704155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:54581)
0806 06:49:47.704289 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:47.704325 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260806 06:49:48.038122 23908 raft_consensus.cc:482] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:48.038220 23908 raft_consensus.cc:504] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5499c0719d1456db7c5c855364a3f17" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 43501 } } peers { permanent_uuid: "80ec2bbbc15d4db9b3b83d85b8eb3eaa" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 35799 } } peers { permanent_uuid: "d814d0e42586446a89552cbd512ee7b0" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 42079 } }
I20260806 06:49:48.038455 23908 leader_election.cc:288] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501), 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799)
W20260806 06:49:48.038971 23483 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a5499c0719d1456db7c5c855364a3f17 (127.6.57.67:43501): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
W20260806 06:49:48.039053 23484 leader_election.cc:341] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 80ec2bbbc15d4db9b3b83d85b8eb3eaa (127.6.57.66:35799): Not found: Tablet not found: 9009d8728e16429a8ded30565831e8ee
I20260806 06:49:48.039101 23484 leader_election.cc:302] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d814d0e42586446a89552cbd512ee7b0; no voters: 80ec2bbbc15d4db9b3b83d85b8eb3eaa, a5499c0719d1456db7c5c855364a3f17
I20260806 06:49:48.039177 23908 raft_consensus.cc:2726] T 9009d8728e16429a8ded30565831e8ee P d814d0e42586446a89552cbd512ee7b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 06:49:48.502521 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:48.502225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:59881)
0806 06:49:48.502399 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:48.502435 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260806 06:49:48.587599 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:48.587373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:33901)
0806 06:49:48.587523 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:48.587559 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260806 06:49:48.705077 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:48.704833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:47629)
0806 06:49:48.704965 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:48.705006 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":58,"threads_started":1}
I20260806 06:49:49.503180 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:49.502966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:42399)
0806 06:49:49.503098 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:49.503133 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260806 06:49:49.588316 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:49.588107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:34947)
0806 06:49:49.588238 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:49.588272 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 06:49:49.705770 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:49.705539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:42113)
0806 06:49:49.705677 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:49.705717 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 06:49:50.503867 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:50.503651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:35887)
0806 06:49:50.503783 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:50.503819 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260806 06:49:50.588932 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:50.588734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:57385)
0806 06:49:50.588862 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:50.588893 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 06:49:50.706409 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:50.706207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:39933)
0806 06:49:50.706332 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:50.706367 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260806 06:49:51.504642 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:51.504437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:39285)
0806 06:49:51.504561 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:51.504595 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260806 06:49:51.589524 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:51.589348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:36755)
0806 06:49:51.589452 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:51.589482 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260806 06:49:51.707166 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:51.706857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:36271)
0806 06:49:51.707009 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:51.707098 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260806 06:49:52.505228 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:52.505034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:59901)
0806 06:49:52.505150 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:52.505181 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260806 06:49:52.590183 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:52.589929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:55423)
0806 06:49:52.590096 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:52.590134 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260806 06:49:52.707816 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:52.707617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:45127)
0806 06:49:52.707738 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:52.707774 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260806 06:49:53.505972 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:53.505764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:46813)
0806 06:49:53.505894 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:53.505925 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 06:49:53.590863 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:53.590670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:53863)
0806 06:49:53.590792 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:53.590823 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260806 06:49:53.708467 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:53.708264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:36309)
0806 06:49:53.708389 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:53.708422 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260806 06:49:54.506811 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:54.506565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.65:48819)
0806 06:49:54.506699 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:54.506737 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260806 06:49:54.591535 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:54.591320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.66:40675)
0806 06:49:54.591462 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:54.591494 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260806 06:49:54.704902 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23429
I20260806 06:49:54.709179 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:54.708974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:41151 (local address 127.6.57.67:43217)
0806 06:49:54.709114 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:54.709143 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.126:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260806 06:49:54.711621 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23602
I20260806 06:49:54.717585 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23741
2026-08-06T06:49:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58171 ms)
[----------] 30 tests from SecurityITest (231538 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:49:54 dist-test-slave-5l3k krb5kdc[23932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:49:54 dist-test-slave-5l3k krb5kdc[23932](info): set up 2 sockets
Aug 06 06:49:54 dist-test-slave-5l3k krb5kdc[23932](info): commencing operation
krb5kdc: starting...
W20260806 06:49:55.776741 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:49:55 dist-test-slave-5l3k krb5kdc[23932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:49:55.897564 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44875
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:44875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:55.981707 23945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:55.981940 23945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:55.981966 23945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:55.983331 23945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:55.983379 23945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:55.983397 23945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:55.983460 23945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:55.983489 23945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:55.985138 23945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:44875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:44875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:55.985440 23945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:55.985694 23945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:55.991012 23953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:55.991106 23950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:55.991124 23945 server_base.cc:1034] running on GCE node
W20260806 06:49:55.991044 23951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:55.991482 23945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:55.991786 23945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:55.992923 23945 hybrid_clock.cc:648] HybridClock initialized: now 1785998995992905 us; error 34 us; skew 500 ppm
Aug 06 06:49:55 dist-test-slave-5l3k krb5kdc[23932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998995, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:55.995361 23945 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:49:55.995707 23945 webserver.cc:460] Webserver started at http://127.6.57.126:39251/ using document root <none> and password file <none>
I20260806 06:49:55.995923 23945 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:55.995965 23945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:55.996047 23945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:55.997040 23945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "bbd0059264864b159dd60fa6f41430a3"
format_stamp: "Formatted at 2026-08-06 06:49:55 on dist-test-slave-5l3k"
server_key: "6a13482b588c7a5a2277d7bc3b3fe9d5"
server_key_iv: "149e7b75a80db3b56ac6d64f238aff94"
server_key_version: "encryptionkey@0"
I20260806 06:49:55.997337 23945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "bbd0059264864b159dd60fa6f41430a3"
format_stamp: "Formatted at 2026-08-06 06:49:55 on dist-test-slave-5l3k"
server_key: "6a13482b588c7a5a2277d7bc3b3fe9d5"
server_key_iv: "149e7b75a80db3b56ac6d64f238aff94"
server_key_version: "encryptionkey@0"
I20260806 06:49:55.998605 23945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:49:55.999325 23960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:55.999517 23945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:55.999608 23945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "bbd0059264864b159dd60fa6f41430a3"
format_stamp: "Formatted at 2026-08-06 06:49:55 on dist-test-slave-5l3k"
server_key: "6a13482b588c7a5a2277d7bc3b3fe9d5"
server_key_iv: "149e7b75a80db3b56ac6d64f238aff94"
server_key_version: "encryptionkey@0"
I20260806 06:49:55.999662 23945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.013914 23945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.015820 23945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.015961 23945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.020174 23945 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:44875
I20260806 06:49:56.020247 24016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:44875 every 8 connection(s)
I20260806 06:49:56.020591 23945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:56.021474 24017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:56.021948 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 23945
I20260806 06:49:56.022066 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:49:56.022280 6373 external_mini_cluster.cc:1428] Setting key 4039620172a65070085dfd961115c3ff
I20260806 06:49:56.024715 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: Bootstrap starting.
I20260806 06:49:56.025393 24017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:56.025720 24017 log.cc:824] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:56.026538 24017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: No bootstrap required, opened a new log
I20260806 06:49:56.028673 24017 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } }
I20260806 06:49:56.028800 24017 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:56.028834 24017 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd0059264864b159dd60fa6f41430a3, State: Initialized, Role: FOLLOWER
I20260806 06:49:56.028960 24017 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } }
I20260806 06:49:56.029033 24017 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:56.029055 24017 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:56.029121 24017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:56.029646 24017 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } }
I20260806 06:49:56.029767 24017 leader_election.cc:302] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbd0059264864b159dd60fa6f41430a3; no voters:
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:56.029960 24017 leader_election.cc:288] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:49:56.029989 24022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:56.030171 24022 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [term 1 LEADER]: Becoming Leader. State: Replica: bbd0059264864b159dd60fa6f41430a3, State: Running, Role: LEADER
I20260806 06:49:56.030213 24017 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:56.031721 24022 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } }
I20260806 06:49:56.032366 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd0059264864b159dd60fa6f41430a3. Latest consensus state: current_term: 1 leader_uuid: "bbd0059264864b159dd60fa6f41430a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } } }
I20260806 06:49:56.032488 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:56.032769 24037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:56.032790 24024 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbd0059264864b159dd60fa6f41430a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd0059264864b159dd60fa6f41430a3" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 44875 } } }
I20260806 06:49:56.032886 24024 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3 [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:56.033244 24037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:56.034814 24037 catalog_manager.cc:1347] Generated new cluster ID: 0aaf1e5250df40098fd1576cc1b0c11f
I20260806 06:49:56.034881 24037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:56.035053 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.022855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.6.57.126:44875)
0806 06:49:56.023100 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.023120 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.023151 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:49:56.023689 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.023698 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.023756 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.023906 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.024529 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.025903 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.026964 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.027568 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.027693 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.030697 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.030701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.030702 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:56.030714 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.033038 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.033426 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.033431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.033431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.033551 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.033835 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.033836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.033837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.034160 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.034305 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.034577 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.034723 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":73,"threads_started":1}
I20260806 06:49:56.043108 24037 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:49:56.043838 24037 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:56.048182 24037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: Generated new TSK 0
I20260806 06:49:56.048395 24037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:49:56.082832 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:56.180984 24043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:56.181211 24043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:56.181252 24043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:56.182374 24043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:56.182441 24043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:56.182487 24043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:56.183957 24043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:56.184212 24043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:56.184437 24043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:56.190591 24052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.190596 24049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.190857 24043 server_base.cc:1034] running on GCE node
W20260806 06:49:56.190660 24048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.191185 24043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:56.191454 24043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:56.192615 24043 hybrid_clock.cc:648] HybridClock initialized: now 1785998996192588 us; error 41 us; skew 500 ppm
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:56.195267 24043 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:49:56.195788 24043 webserver.cc:460] Webserver started at http://127.6.57.65:45631/ using document root <none> and password file <none>
I20260806 06:49:56.196053 24043 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:56.196113 24043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:56.196246 24043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:56.197654 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "1b62e89af86ef86eece67d40bb7e348a"
server_key_iv: "41f91b1d4809d903c5c39bd958117621"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.198130 24043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "1b62e89af86ef86eece67d40bb7e348a"
server_key_iv: "41f91b1d4809d903c5c39bd958117621"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.199862 24043 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 06:49:56.200870 24059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.201092 24043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:49:56.201215 24043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "1b62e89af86ef86eece67d40bb7e348a"
server_key_iv: "41f91b1d4809d903c5c39bd958117621"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.201308 24043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.217820 24043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.219712 24043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.219823 24043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.219997 24043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:56.220330 24043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:56.220366 24043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.220388 24043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:56.220444 24043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.226032 24043 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:44117
I20260806 06:49:56.226120 24178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:44117 every 8 connection(s)
I20260806 06:49:56.226430 24043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:56.228310 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24043
I20260806 06:49:56.228440 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:49:56.228680 6373 external_mini_cluster.cc:1428] Setting key 3148c2b0d244d244c6cc576a91541ea0
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:56.236492 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.227156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:51927 (local address 127.6.57.126:44875)
0806 06:49:56.227195 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.227197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.227411 (+ 214us) server_negotiation.cc:407] Connection header received
0806 06:49:56.227933 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.227934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.227947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.227997 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.229194 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.229688 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.230783 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.231023 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.231104 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.233537 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.233539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.233539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:56.233548 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.235135 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.235507 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.235508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.235509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.235579 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.235823 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.235824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.235825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.235972 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.236050 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.236295 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.236392 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:49:56.236501 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.227148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44875 (local address 127.6.57.65:51927)
0806 06:49:56.227294 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.227329 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.227820 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.228070 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.228073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.228877 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:56.229080 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.229085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.229787 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.229788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.230690 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.230693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.231133 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.231135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.231310 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.231909 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:56.231918 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:56.233441 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:56.235221 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.235227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.235229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.235434 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.235647 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.235647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.235648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.235736 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.236043 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:56.236048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:56.236162 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.236241 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.236354 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260806 06:49:56.237043 24179 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44875
I20260806 06:49:56.237131 24179 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:56.237310 24179 heartbeater.cc:511] Master 127.6.57.126:44875 requested a full tablet report, sending...
I20260806 06:49:56.237833 23977 ts_manager.cc:194] Registered new tserver with Master: 6c4f6f80a9d74298b7c3aa2a0b4d4df4 (127.6.57.65:44117)
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:49:56.238426 23977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:51927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:49:56.256706 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:56.337641 24184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:56.337882 24184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:56.337924 24184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:56.339124 24184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:56.339188 24184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:56.339234 24184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:56.340698 24184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:56.340982 24184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:56.341207 24184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:56.346002 24189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.346078 24190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.346014 24192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.346215 24184 server_base.cc:1034] running on GCE node
I20260806 06:49:56.346575 24184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:56.346843 24184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:56.347999 24184 hybrid_clock.cc:648] HybridClock initialized: now 1785998996347965 us; error 53 us; skew 500 ppm
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:56.350454 24184 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:49:56.350869 24184 webserver.cc:460] Webserver started at http://127.6.57.66:37805/ using document root <none> and password file <none>
I20260806 06:49:56.351058 24184 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:56.351100 24184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:56.351187 24184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:56.352137 24184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "9cf2b18c12744b5a9126ce0060bcbfb5"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "8cb7766218e62fc9731f0e5b5b310b6a"
server_key_iv: "64e3bf89f5a17ec356415dc2a1ec0ab1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.352417 24184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "9cf2b18c12744b5a9126ce0060bcbfb5"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "8cb7766218e62fc9731f0e5b5b310b6a"
server_key_iv: "64e3bf89f5a17ec356415dc2a1ec0ab1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.353592 24184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:49:56.354236 24200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.354383 24184 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:56.354468 24184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "9cf2b18c12744b5a9126ce0060bcbfb5"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "8cb7766218e62fc9731f0e5b5b310b6a"
server_key_iv: "64e3bf89f5a17ec356415dc2a1ec0ab1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.354575 24184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.367194 24184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.368963 24184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.369077 24184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.369257 24184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:56.369567 24184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:56.369602 24184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.369630 24184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:56.369676 24184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.375284 24184 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:39945
I20260806 06:49:56.375391 24317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:39945 every 8 connection(s)
I20260806 06:49:56.375717 24184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:49:56.381182 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24184
I20260806 06:49:56.381274 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:56.381491 6373 external_mini_cluster.cc:1428] Setting key a69d5c4832cc05e359352471711b2140
I20260806 06:49:56.384819 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.376427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:33515 (local address 127.6.57.126:44875)
0806 06:49:56.376509 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.376512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.376724 (+ 212us) server_negotiation.cc:407] Connection header received
0806 06:49:56.377180 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.377181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.377192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.377240 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.378149 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.378668 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.379616 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.379805 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.379874 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.381962 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.381964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.381964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:56.381972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.383425 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.383849 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.383850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.383851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.383912 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.384168 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.384169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.384170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.384289 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.384367 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.384638 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.384716 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 06:49:56.385021 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.376394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44875 (local address 127.6.57.66:33515)
0806 06:49:56.376599 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.376639 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.377081 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.377299 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.377303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.377874 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:56.378057 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.378062 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.378741 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.378741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.379519 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.379522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.379888 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.379889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.379982 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.380416 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:56.380424 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:56.381859 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:56.383516 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.383523 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.383526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.383749 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.383981 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.383982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.383983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.384097 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.384358 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:56.384365 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:56.384502 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.384605 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.384760 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":128,"threads_started":1}
I20260806 06:49:56.385511 24318 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44875
I20260806 06:49:56.385634 24318 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:56.385808 24318 heartbeater.cc:511] Master 127.6.57.126:44875 requested a full tablet report, sending...
I20260806 06:49:56.386250 23977 ts_manager.cc:194] Registered new tserver with Master: 9cf2b18c12744b5a9126ce0060bcbfb5 (127.6.57.66:39945)
I20260806 06:49:56.386842 23977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:33515
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:49:56.409385 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:56.489326 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:56.489509 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:56.489531 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:56.490661 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:56.490700 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:56.490751 24323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:56.492182 24323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:44875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:56.492411 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:56.492643 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:56.497375 24329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.497381 24331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.497408 24328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.497778 24323 server_base.cc:1034] running on GCE node
I20260806 06:49:56.497999 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:56.498246 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:56.499390 24323 hybrid_clock.cc:648] HybridClock initialized: now 1785998996499390 us; error 41 us; skew 500 ppm
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:56.501837 24323 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:49:56.502277 24323 webserver.cc:460] Webserver started at http://127.6.57.67:42707/ using document root <none> and password file <none>
I20260806 06:49:56.502535 24323 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:56.502597 24323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:56.502687 24323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:56.503682 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "09873a47ab1442f881724b7074e1b53a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "63637c40c98ae22e74c09e2445dcebe4"
server_key_iv: "5cbe228e456ca07adc8a61193ae4979b"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.503976 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "09873a47ab1442f881724b7074e1b53a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "63637c40c98ae22e74c09e2445dcebe4"
server_key_iv: "5cbe228e456ca07adc8a61193ae4979b"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.505203 24323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:49:56.505872 24338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.506029 24323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:49:56.506103 24323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "09873a47ab1442f881724b7074e1b53a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "63637c40c98ae22e74c09e2445dcebe4"
server_key_iv: "5cbe228e456ca07adc8a61193ae4979b"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.506152 24323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.514333 24323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.516027 24323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.516132 24323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.516291 24323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:56.516513 24323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:56.516542 24323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.516569 24323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:56.516583 24323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.522324 24323 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:45907
I20260806 06:49:56.522385 24453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:45907 every 8 connection(s)
I20260806 06:49:56.522826 24323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:49:56.523808 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24323
I20260806 06:49:56.523882 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:49:56.524076 6373 external_mini_cluster.cc:1428] Setting key 4949566ae3a0c8045eeab40e6ff6c1ce
Aug 06 06:49:56 dist-test-slave-5l3k krb5kdc[23932](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998996, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:56.531670 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.523469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:58671 (local address 127.6.57.126:44875)
0806 06:49:56.523539 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.523541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.523698 (+ 157us) server_negotiation.cc:407] Connection header received
0806 06:49:56.524208 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.524209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.524221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.524269 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.525222 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.525696 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.526747 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.526962 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.527048 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.529009 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.529010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.529011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:56.529017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.530274 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.530739 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.530740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.530741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.530796 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.531083 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.531084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.531084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.531210 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.531278 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.531547 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.531612 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:49:56.531831 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.523429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:44875 (local address 127.6.57.67:58671)
0806 06:49:56.523581 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.523611 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.524089 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.524328 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.524332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.524969 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:56.525135 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.525140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.525792 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.525794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.526661 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.526663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.527078 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.527079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.527201 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.527684 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:56.527691 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:56.528913 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:56.530386 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.530393 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.530395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.530656 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.530860 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:56.530861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:56.530862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:56.531013 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:56.531283 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:56.531289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:56.531410 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.531511 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.531635 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260806 06:49:56.532282 24454 heartbeater.cc:347] Connected to a master server at 127.6.57.126:44875
I20260806 06:49:56.532392 24454 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:56.532546 24454 heartbeater.cc:511] Master 127.6.57.126:44875 requested a full tablet report, sending...
I20260806 06:49:56.532924 23977 ts_manager.cc:194] Registered new tserver with Master: 09873a47ab1442f881724b7074e1b53a (127.6.57.67:45907)
I20260806 06:49:56.533265 23977 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:58671
I20260806 06:49:56.535621 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:49:56.542533 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.536832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.6.57.126:44875)
0806 06:49:56.536909 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.536911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.537017 (+ 106us) server_negotiation.cc:407] Connection header received
0806 06:49:56.537089 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.537090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.537099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.537172 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.537757 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.538207 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.538983 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.539168 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.539247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.540253 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.540254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.540255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:56.540261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.541381 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.541706 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.541708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.541708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.541763 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:56.541964 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:56.541965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:56.541966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:56.542117 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.542218 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.542328 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.542408 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 06:49:56.543962 23977 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:49:56.545503 23977 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:49:56.551959 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:39945 (local address 127.0.0.1:40424)
0806 06:49:56.548235 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.548242 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.548306 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.548924 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.548925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.548927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:56.548976 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.548978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.550000 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.550001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.550890 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.550893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551616 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.551697 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551710 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.551813 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.551869 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260806 06:49:56.552057 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.6.57.65:44117)
0806 06:49:56.548681 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.548686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.548694 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:49:56.548729 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.548731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.548745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.548821 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:56.548936 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.549704 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.550992 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551720 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551799 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551846 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.551942 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.551975 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":81,"threads_started":1}
I20260806 06:49:56.552067 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:45907 (local address 127.0.0.1:48398)
0806 06:49:56.547859 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.547896 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.547960 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.548167 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.548170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.548172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:56.548218 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.548219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.549513 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.549515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.550814 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.550817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551806 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.551878 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551881 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.551935 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.551976 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":35,"spinlock_wait_cycles":384,"thread_start_us":54,"threads_started":1}
I20260806 06:49:56.552081 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44117 (local address 127.0.0.1:58330)
0806 06:49:56.548074 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.548083 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.548122 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.548826 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.548827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.548829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:56.548861 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.548862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.549832 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.549832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.550911 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.550913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551809 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.551875 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551878 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.551929 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.551964 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":8,"thread_start_us":52,"threads_started":1}
I20260806 06:49:56.552309 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.6.57.67:45907)
0806 06:49:56.548012 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.548016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.548025 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:56.548060 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.548062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.548075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.548156 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:56.548301 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.549412 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.550909 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551627 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551698 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551756 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.552181 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.552229 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260806 06:49:56.552809 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 1848046913274f258a1dfa24957cbd89 (DEFAULT_TABLE table=test-table [id=01b731c2ba574ef6bbaa048d79ecad5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:56.552913 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.547929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.6.57.66:39945)
0806 06:49:56.548445 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.548449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.548459 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:49:56.548494 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.548496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.548510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.549051 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:56.549118 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.549902 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.550981 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.551528 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.551616 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.551678 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.552788 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.552832 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":91,"threads_started":1}
I20260806 06:49:56.552966 24388 tablet_service.cc:1467] Processing CreateTablet for tablet 1848046913274f258a1dfa24957cbd89 (DEFAULT_TABLE table=test-table [id=01b731c2ba574ef6bbaa048d79ecad5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:56.553167 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1848046913274f258a1dfa24957cbd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:56.553372 24388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1848046913274f258a1dfa24957cbd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:56.553583 24250 tablet_service.cc:1467] Processing CreateTablet for tablet 1848046913274f258a1dfa24957cbd89 (DEFAULT_TABLE table=test-table [id=01b731c2ba574ef6bbaa048d79ecad5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:56.553890 24250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1848046913274f258a1dfa24957cbd89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:56.556255 24468 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Bootstrap starting.
I20260806 06:49:56.556790 24469 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: Bootstrap starting.
I20260806 06:49:56.557014 24468 tablet_bootstrap.cc:654] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:56.557337 24470 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Bootstrap starting.
I20260806 06:49:56.557597 24469 tablet_bootstrap.cc:654] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:56.558020 24470 tablet_bootstrap.cc:654] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:56.558076 24468 log.cc:824] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:56.558391 24469 log.cc:824] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:56.558712 24470 log.cc:824] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:56.559023 24468 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: No bootstrap required, opened a new log
I20260806 06:49:56.559108 24468 ts_tablet_manager.cc:1397] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:49:56.559448 24470 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: No bootstrap required, opened a new log
I20260806 06:49:56.559531 24470 ts_tablet_manager.cc:1397] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:56.560559 24468 raft_consensus.cc:348] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.560701 24468 raft_consensus.cc:374] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:56.560726 24468 raft_consensus.cc:729] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09873a47ab1442f881724b7074e1b53a, State: Initialized, Role: FOLLOWER
I20260806 06:49:56.560855 24468 consensus_queue.cc:260] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.560920 24470 raft_consensus.cc:348] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.561128 24470 raft_consensus.cc:374] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:56.561182 24470 raft_consensus.cc:729] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cf2b18c12744b5a9126ce0060bcbfb5, State: Initialized, Role: FOLLOWER
I20260806 06:49:56.561265 24468 ts_tablet_manager.cc:1428] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:56.561321 24470 consensus_queue.cc:260] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.561398 24454 heartbeater.cc:503] Master 127.6.57.126:44875 was elected leader, sending a full tablet report...
I20260806 06:49:56.561594 24470 ts_tablet_manager.cc:1428] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:56.561622 24318 heartbeater.cc:503] Master 127.6.57.126:44875 was elected leader, sending a full tablet report...
I20260806 06:49:56.561981 24469 tablet_bootstrap.cc:492] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: No bootstrap required, opened a new log
I20260806 06:49:56.562057 24469 ts_tablet_manager.cc:1397] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 06:49:56.563688 24469 raft_consensus.cc:348] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.563813 24469 raft_consensus.cc:374] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:56.563835 24469 raft_consensus.cc:729] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c4f6f80a9d74298b7c3aa2a0b4d4df4, State: Initialized, Role: FOLLOWER
I20260806 06:49:56.563951 24469 consensus_queue.cc:260] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.564218 24469 ts_tablet_manager.cc:1428] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:56.564277 24179 heartbeater.cc:503] Master 127.6.57.126:44875 was elected leader, sending a full tablet report...
I20260806 06:49:56.616442 24475 raft_consensus.cc:482] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:56.616561 24475 raft_consensus.cc:504] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.616983 24475 leader_election.cc:288] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c4f6f80a9d74298b7c3aa2a0b4d4df4 (127.6.57.65:44117), 9cf2b18c12744b5a9126ce0060bcbfb5 (127.6.57.66:39945)
I20260806 06:49:56.620770 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.617236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:52323 (local address 127.6.57.65:44117)
0806 06:49:56.617280 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.617282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.617293 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:49:56.617402 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.617403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.617416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.617486 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:56.617664 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.618415 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.619819 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.620341 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.620432 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.620442 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.620665 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.620703 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:49:56.620770 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.617299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:49173 (local address 127.6.57.66:39945)
0806 06:49:56.617369 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.617371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.617543 (+ 172us) server_negotiation.cc:407] Connection header received
0806 06:49:56.617593 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.617594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.617604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.617677 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:56.617825 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.618524 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.619818 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.620345 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.620432 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.620442 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.620665 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.620703 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:49:56.620820 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.617174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44117 (local address 127.6.57.67:52323)
0806 06:49:56.617242 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.617254 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.617299 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.617502 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.617503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.617505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:56.617581 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.617583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.618535 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.618537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.619713 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.619716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.620468 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.620470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.620562 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.620567 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.620642 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.620698 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":10}
I20260806 06:49:56.620841 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.617246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:39945 (local address 127.6.57.67:49173)
0806 06:49:56.617444 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.617452 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.617532 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.617691 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.617692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.617693 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:56.617756 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.617757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.618578 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.618579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.619713 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.619716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.620469 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.620470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.620562 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.620567 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.620642 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.620697 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":16,"thread_start_us":145,"threads_started":1}
I20260806 06:49:56.621222 24270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1848046913274f258a1dfa24957cbd89" candidate_uuid: "09873a47ab1442f881724b7074e1b53a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" is_pre_election: true
I20260806 06:49:56.621236 24131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1848046913274f258a1dfa24957cbd89" candidate_uuid: "09873a47ab1442f881724b7074e1b53a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" is_pre_election: true
I20260806 06:49:56.621366 24270 raft_consensus.cc:2454] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09873a47ab1442f881724b7074e1b53a in term 0.
I20260806 06:49:56.621388 24131 raft_consensus.cc:2454] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09873a47ab1442f881724b7074e1b53a in term 0.
I20260806 06:49:56.621609 24341 leader_election.cc:302] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09873a47ab1442f881724b7074e1b53a, 9cf2b18c12744b5a9126ce0060bcbfb5; no voters:
I20260806 06:49:56.621790 24475 raft_consensus.cc:2781] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:49:56.621873 24475 raft_consensus.cc:482] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:49:56.621904 24475 raft_consensus.cc:3037] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:56.622689 24475 raft_consensus.cc:504] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.622843 24475 leader_election.cc:288] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [CANDIDATE]: Term 1 election: Requested vote from peers 6c4f6f80a9d74298b7c3aa2a0b4d4df4 (127.6.57.65:44117), 9cf2b18c12744b5a9126ce0060bcbfb5 (127.6.57.66:39945)
I20260806 06:49:56.623013 24131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1848046913274f258a1dfa24957cbd89" candidate_uuid: "09873a47ab1442f881724b7074e1b53a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4"
I20260806 06:49:56.623023 24270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1848046913274f258a1dfa24957cbd89" candidate_uuid: "09873a47ab1442f881724b7074e1b53a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5"
I20260806 06:49:56.623099 24131 raft_consensus.cc:3037] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:56.623095 24270 raft_consensus.cc:3037] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:56.623793 24270 raft_consensus.cc:2454] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09873a47ab1442f881724b7074e1b53a in term 1.
I20260806 06:49:56.623804 24131 raft_consensus.cc:2454] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09873a47ab1442f881724b7074e1b53a in term 1.
I20260806 06:49:56.623947 24341 leader_election.cc:302] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09873a47ab1442f881724b7074e1b53a, 9cf2b18c12744b5a9126ce0060bcbfb5; no voters:
I20260806 06:49:56.624033 24475 raft_consensus.cc:2781] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:56.624245 24475 raft_consensus.cc:686] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 1 LEADER]: Becoming Leader. State: Replica: 09873a47ab1442f881724b7074e1b53a, State: Running, Role: LEADER
I20260806 06:49:56.624358 24475 consensus_queue.cc:237] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } }
I20260806 06:49:56.625197 23976 catalog_manager.cc:5495] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a reported cstate change: term changed from 0 to 1, leader changed from <none> to 09873a47ab1442f881724b7074e1b53a (127.6.57.67). New cstate: current_term: 1 leader_uuid: "09873a47ab1442f881724b7074e1b53a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09873a47ab1442f881724b7074e1b53a" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 } health_report { overall_health: UNKNOWN } } }
W20260806 06:49:56.626305 24319 tablet.cc:2369] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:49:56.644791 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.642561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.6.57.67:45907)
0806 06:49:56.642609 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.642611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.642618 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:49:56.642643 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.642644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.642659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.642734 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:56.642871 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.643308 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.644049 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.644234 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.644307 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.644475 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:56.644561 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:49:56.644649 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.644671 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.644690 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:49:56.646463 24270 raft_consensus.cc:1261] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 1 FOLLOWER]: Refusing update from remote peer 09873a47ab1442f881724b7074e1b53a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:56.646575 24131 raft_consensus.cc:1261] T 1848046913274f258a1dfa24957cbd89 P 6c4f6f80a9d74298b7c3aa2a0b4d4df4 [term 1 FOLLOWER]: Refusing update from remote peer 09873a47ab1442f881724b7074e1b53a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:56.646881 24475 consensus_queue.cc:1035] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cf2b18c12744b5a9126ce0060bcbfb5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 39945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:56.647100 24479 consensus_queue.cc:1035] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c4f6f80a9d74298b7c3aa2a0b4d4df4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 44117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:56.649724 24483 mvcc.cc:205] Tried to move back new op lower bound from 7315451890261454848 to 7315451890174107648. Current Snapshot: MvccSnapshot[applied={T|T < 7315451890261454848}]
I20260806 06:49:56.651539 24484 mvcc.cc:205] Tried to move back new op lower bound from 7315451890261454848 to 7315451890174107648. Current Snapshot: MvccSnapshot[applied={T|T < 7315451890261454848 or (T in {7315451890261454848})}]
I20260806 06:49:56.651824 23977 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:56.651925 23977 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:56.652678 23977 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bbd0059264864b159dd60fa6f41430a3: Sending DeleteTablet for 3 replicas of tablet 1848046913274f258a1dfa24957cbd89
I20260806 06:49:56.653015 24388 tablet_service.cc:1514] Processing DeleteTablet for tablet 1848046913274f258a1dfa24957cbd89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:56 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:48398
I20260806 06:49:56.653103 24250 tablet_service.cc:1514] Processing DeleteTablet for tablet 1848046913274f258a1dfa24957cbd89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:56 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260806 06:49:56.653101 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 1848046913274f258a1dfa24957cbd89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:56 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:58330
I20260806 06:49:56.653313 24493 tablet_replica.cc:330] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: stopping tablet replica
I20260806 06:49:56.653443 24493 raft_consensus.cc:2229] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:56.653539 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24043
I20260806 06:49:56.653570 24493 raft_consensus.cc:2258] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:56.654579 24493 ts_tablet_manager.cc:1905] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:56.656330 24493 ts_tablet_manager.cc:1918] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:56.656405 24493 log.cc:1196] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/1848046913274f258a1dfa24957cbd89
I20260806 06:49:56.656661 24493 ts_tablet_manager.cc:1939] T 1848046913274f258a1dfa24957cbd89 P 9cf2b18c12744b5a9126ce0060bcbfb5: Deleting consensus metadata
I20260806 06:49:56.657300 23963 catalog_manager.cc:4841] TS 9cf2b18c12744b5a9126ce0060bcbfb5 (127.6.57.66:39945): tablet 1848046913274f258a1dfa24957cbd89 (table test-table [id=01b731c2ba574ef6bbaa048d79ecad5d]) successfully deleted
I20260806 06:49:56.658229 24492 tablet_replica.cc:330] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: stopping tablet replica
I20260806 06:49:56.658373 24492 raft_consensus.cc:2229] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:49:56.658510 24492 raft_consensus.cc:2258] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:56.658885 24492 ts_tablet_manager.cc:1905] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:56.660568 24492 ts_tablet_manager.cc:1918] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:56.660673 24492 log.cc:1196] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/1848046913274f258a1dfa24957cbd89
I20260806 06:49:56.660943 24492 ts_tablet_manager.cc:1939] T 1848046913274f258a1dfa24957cbd89 P 09873a47ab1442f881724b7074e1b53a: Deleting consensus metadata
I20260806 06:49:56.661341 23961 catalog_manager.cc:4841] TS 09873a47ab1442f881724b7074e1b53a (127.6.57.67:45907): tablet 1848046913274f258a1dfa24957cbd89 (table test-table [id=01b731c2ba574ef6bbaa048d79ecad5d]) successfully deleted
W20260806 06:49:56.663515 23964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:44117 (error 108)
I20260806 06:49:56.663792 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24184
I20260806 06:49:56.667902 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.667762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:44117 (local address 127.0.0.1:58334)
0806 06:49:56.667833 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.667875 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:44117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 06:49:56.668053 23964 catalog_manager.cc:4584] TS 6c4f6f80a9d74298b7c3aa2a0b4d4df4 (127.6.57.65:44117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1848046913274f258a1dfa24957cbd89: Network error: Client connection negotiation failed: client connection to 127.6.57.65:44117: connect: Connection refused (error 111)
I20260806 06:49:56.673594 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24323
I20260806 06:49:56.680665 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 23945
2026-08-06T06:49:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1962 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T06:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:49:56Z Disabled control of system clock
I20260806 06:49:56.709121 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:39329
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:39329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:49:56.792304 24499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:56.792523 24499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:56.792551 24499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:56.793741 24499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:56.793782 24499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:56.793795 24499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:56.793807 24499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:56.793819 24499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:56.795148 24499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:39329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:56.795390 24499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:56.795675 24499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:56.800662 24504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.800663 24507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.800768 24505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.801208 24499 server_base.cc:1034] running on GCE node
I20260806 06:49:56.801465 24499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:56.801779 24499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:56.802942 24499 hybrid_clock.cc:648] HybridClock initialized: now 1785998996802930 us; error 55 us; skew 500 ppm
I20260806 06:49:56.804077 24499 webserver.cc:460] Webserver started at http://127.6.57.126:36643/ using document root <none> and password file <none>
I20260806 06:49:56.804280 24499 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:56.804320 24499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:56.804409 24499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:56.805349 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "fdb5ded5dba748599a1bdec44e6c362a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "a4adb3c59d8c02f44b1d694c76940184"
server_key_iv: "6c0fa00c8e5b455add2579381dec29b1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.805650 24499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "fdb5ded5dba748599a1bdec44e6c362a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "a4adb3c59d8c02f44b1d694c76940184"
server_key_iv: "6c0fa00c8e5b455add2579381dec29b1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.806955 24499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:49:56.807675 24513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.807852 24499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:49:56.807945 24499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "fdb5ded5dba748599a1bdec44e6c362a"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "a4adb3c59d8c02f44b1d694c76940184"
server_key_iv: "6c0fa00c8e5b455add2579381dec29b1"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.808048 24499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.821688 24499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.822008 24499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.822176 24499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.826442 24499 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:39329
I20260806 06:49:56.826553 24565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:39329 every 8 connection(s)
I20260806 06:49:56.826882 24499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:56.827936 24566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:56.830497 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: Bootstrap starting.
I20260806 06:49:56.831169 24566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:56.831422 24566 log.cc:824] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:56.832078 24566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: No bootstrap required, opened a new log
I20260806 06:49:56.833585 24566 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } }
I20260806 06:49:56.833699 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24499
I20260806 06:49:56.833737 24566 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:56.833761 24566 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb5ded5dba748599a1bdec44e6c362a, State: Initialized, Role: FOLLOWER
I20260806 06:49:56.833803 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:49:56.833935 24566 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } }
I20260806 06:49:56.834025 24566 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:56.834062 6373 external_mini_cluster.cc:1428] Setting key 8e8799efb7a628de613743665cbe2bae
I20260806 06:49:56.834082 24566 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:56.834128 24566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:56.834844 24566 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } }
I20260806 06:49:56.834966 24566 leader_election.cc:302] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdb5ded5dba748599a1bdec44e6c362a; no voters:
I20260806 06:49:56.835137 24566 leader_election.cc:288] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:49:56.835166 24571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:56.835355 24571 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [term 1 LEADER]: Becoming Leader. State: Replica: fdb5ded5dba748599a1bdec44e6c362a, State: Running, Role: LEADER
I20260806 06:49:56.835456 24566 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:56.835479 24571 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } }
I20260806 06:49:56.835923 24571 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdb5ded5dba748599a1bdec44e6c362a. Latest consensus state: current_term: 1 leader_uuid: "fdb5ded5dba748599a1bdec44e6c362a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } } }
I20260806 06:49:56.835932 24572 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdb5ded5dba748599a1bdec44e6c362a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb5ded5dba748599a1bdec44e6c362a" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 39329 } } }
I20260806 06:49:56.836067 24571 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:56.836295 24572 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:56.836460 24580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:56.836956 24580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:56.838815 24580 catalog_manager.cc:1347] Generated new cluster ID: 75efccfe04724887aefa34cf284663bb
I20260806 06:49:56.838871 24580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:56.840433 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.834611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.6.57.126:39329)
0806 06:49:56.834818 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.834824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.834852 (+ 28us) server_negotiation.cc:407] Connection header received
0806 06:49:56.834940 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.834945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.835033 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.835182 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.835403 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.837142 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.838190 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.838798 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.838925 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.839676 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.839680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.839681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:56.839909 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.839951 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:56.839966 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.840060 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.840132 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.840212 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260806 06:49:56.858435 24580 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:49:56.859176 24580 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:56.868774 24580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: Generated new TSK 0
I20260806 06:49:56.868965 24580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:49:56.871338 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--builtin_ntp_servers=127.6.57.84:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:49:56.953194 24590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:56.953429 24590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:56.953465 24590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:56.954635 24590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:56.954689 24590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:56.954731 24590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:56.956065 24590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:56.956327 24590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:56.956631 24590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:56.962627 24596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.962759 24590 server_base.cc:1034] running on GCE node
W20260806 06:49:56.962841 24599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:56.962654 24595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:56.963151 24590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:56.963402 24590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:56.964558 24590 hybrid_clock.cc:648] HybridClock initialized: now 1785998996964534 us; error 42 us; skew 500 ppm
I20260806 06:49:56.965806 24590 webserver.cc:460] Webserver started at http://127.6.57.65:44475/ using document root <none> and password file <none>
I20260806 06:49:56.966076 24590 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:56.966127 24590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:56.966261 24590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:56.967381 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "5f3328888665403283d51925209f181f"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "92b7db9a8664a1f511bbb4da4e6df267"
server_key_iv: "b60d54a9faa3495b608a342f3f8549d7"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.967742 24590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "5f3328888665403283d51925209f181f"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "92b7db9a8664a1f511bbb4da4e6df267"
server_key_iv: "b60d54a9faa3495b608a342f3f8549d7"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.969074 24590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:49:56.969779 24606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.969952 24590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:56.970043 24590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "5f3328888665403283d51925209f181f"
format_stamp: "Formatted at 2026-08-06 06:49:56 on dist-test-slave-5l3k"
server_key: "92b7db9a8664a1f511bbb4da4e6df267"
server_key_iv: "b60d54a9faa3495b608a342f3f8549d7"
server_key_version: "encryptionkey@0"
I20260806 06:49:56.970118 24590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:56.978174 24590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:56.978396 24590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:56.978561 24590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:56.978745 24590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:56.978991 24590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:56.979022 24590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.979041 24590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:56.979053 24590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:56.984736 24590 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:43931
I20260806 06:49:56.984812 24723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:43931 every 8 connection(s)
I20260806 06:49:56.985146 24590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:56.986793 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24590
I20260806 06:49:56.987001 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:49:56.987193 6373 external_mini_cluster.cc:1428] Setting key b89df1b0ac4e8bdf3b919ef06447d84d
I20260806 06:49:56.988205 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--builtin_ntp_servers=127.6.57.84:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:49:56.992112 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.985894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:54081 (local address 127.6.57.126:39329)
0806 06:49:56.985981 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:56.985984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:56.986257 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:49:56.987034 (+ 777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:56.987036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:56.987052 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:56.987085 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:56.988142 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.988839 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.990368 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.990733 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.990814 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.991754 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:56.991755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:56.991756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:56.991761 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:56.991778 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:56.991783 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:56.991843 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:56.991950 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:49:56.991986 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:49:56.992583 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:56.985872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:39329 (local address 127.6.57.65:54081)
0806 06:49:56.986106 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:56.986154 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:56.986889 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:56.987273 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:56.987277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:56.987289 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:56.988006 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.988014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.988949 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.988950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.990265 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:56.990268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:56.990834 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:56.990835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:56.990988 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:56.991587 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:56.991587 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:56.991625 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:56.991652 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:56.991865 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:56.991871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:56.991874 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:56.991946 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 06:49:56.992058 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260806 06:49:56.993098 24724 heartbeater.cc:347] Connected to a master server at 127.6.57.126:39329
I20260806 06:49:56.993206 24724 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:56.993382 24724 heartbeater.cc:511] Master 127.6.57.126:39329 requested a full tablet report, sending...
I20260806 06:49:56.993952 24530 ts_manager.cc:194] Registered new tserver with Master: 5f3328888665403283d51925209f181f (127.6.57.65:43931)
I20260806 06:49:56.994989 24530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.65:54081
W20260806 06:49:57.071488 24727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:57.071708 24727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:57.071730 24727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:57.072942 24727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:57.072981 24727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:57.073006 24727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:57.074604 24727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:57.074847 24727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:57.075137 24727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:57.080209 24732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:57.080193 24735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:57.080209 24727 server_base.cc:1034] running on GCE node
W20260806 06:49:57.080193 24733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:57.080667 24727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:57.080907 24727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:57.082180 24727 hybrid_clock.cc:648] HybridClock initialized: now 1785998997082170 us; error 37 us; skew 500 ppm
I20260806 06:49:57.083560 24727 webserver.cc:460] Webserver started at http://127.6.57.66:39965/ using document root <none> and password file <none>
I20260806 06:49:57.083818 24727 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:57.083905 24727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:57.084043 24727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:57.085001 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "a82423cf49f941bfa1045ef6c11503c3"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "0b059a675c0b984bc3edfb153817d837"
server_key_iv: "4005a2a2d0dcf839a7c2598df39df7b2"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.085289 24727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "a82423cf49f941bfa1045ef6c11503c3"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "0b059a675c0b984bc3edfb153817d837"
server_key_iv: "4005a2a2d0dcf839a7c2598df39df7b2"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.086580 24727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:49:57.087286 24742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.087469 24727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:57.087543 24727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "a82423cf49f941bfa1045ef6c11503c3"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "0b059a675c0b984bc3edfb153817d837"
server_key_iv: "4005a2a2d0dcf839a7c2598df39df7b2"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.087594 24727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:57.102906 24727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:57.103124 24727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:57.103219 24727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:57.103374 24727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:57.103626 24727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:57.103655 24727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.103674 24727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:57.103686 24727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.109354 24727 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:46815
I20260806 06:49:57.109409 24861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:46815 every 8 connection(s)
I20260806 06:49:57.109699 24727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:49:57.114468 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24727
I20260806 06:49:57.114621 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:49:57.114892 6373 external_mini_cluster.cc:1428] Setting key 212fb04d7621b261e9c7d13f123df21d
I20260806 06:49:57.115208 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.110449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:39987 (local address 127.6.57.126:39329)
0806 06:49:57.110562 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.110565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.110810 (+ 245us) server_negotiation.cc:407] Connection header received
0806 06:49:57.111296 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.111297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.111311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.111379 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:57.112010 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.112610 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.113522 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.113729 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.113798 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.114910 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:57.114912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:57.114912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:57.114918 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:57.114933 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:57.114937 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:57.115005 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.115085 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.115132 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 06:49:57.115381 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.110436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:39329 (local address 127.6.57.66:39987)
0806 06:49:57.110686 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.110716 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.111185 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.111426 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.111430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.111437 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:57.111926 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.111930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.112697 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.112698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.113424 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.113426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.113933 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.113935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.114048 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.114723 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:57.114723 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:57.114781 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:57.114802 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:57.115008 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:57.115014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:57.115017 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.115086 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.115179 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":157,"threads_started":1}
I20260806 06:49:57.115862 24862 heartbeater.cc:347] Connected to a master server at 127.6.57.126:39329
I20260806 06:49:57.115965 24862 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:57.116146 24862 heartbeater.cc:511] Master 127.6.57.126:39329 requested a full tablet report, sending...
I20260806 06:49:57.116205 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--builtin_ntp_servers=127.6.57.84:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:49:57.116503 24530 ts_manager.cc:194] Registered new tserver with Master: a82423cf49f941bfa1045ef6c11503c3 (127.6.57.66:46815)
I20260806 06:49:57.116892 24530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.66:39987
W20260806 06:49:57.202306 24865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:57.202613 24865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:57.202652 24865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:57.203801 24865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:57.203867 24865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:57.203912 24865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:57.205356 24865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:39329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:57.205612 24865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:57.205909 24865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:57.210843 24870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:57.210836 24871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:57.210891 24865 server_base.cc:1034] running on GCE node
W20260806 06:49:57.210820 24873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:57.211208 24865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:57.211457 24865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:57.212653 24865 hybrid_clock.cc:648] HybridClock initialized: now 1785998997212642 us; error 29 us; skew 500 ppm
I20260806 06:49:57.213867 24865 webserver.cc:460] Webserver started at http://127.6.57.67:42299/ using document root <none> and password file <none>
I20260806 06:49:57.214139 24865 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:57.214186 24865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:57.214350 24865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:57.215440 24865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "c91368cb0cf84a3bad3b6bb92a222f7c"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "c83d74325dc78592324a4ef67c1eed1b"
server_key_iv: "86e643a4a030a6941b0c0757ce358b84"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.215808 24865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "c91368cb0cf84a3bad3b6bb92a222f7c"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "c83d74325dc78592324a4ef67c1eed1b"
server_key_iv: "86e643a4a030a6941b0c0757ce358b84"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.217077 24865 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:49:57.217778 24879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.217958 24865 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:57.218047 24865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "c91368cb0cf84a3bad3b6bb92a222f7c"
format_stamp: "Formatted at 2026-08-06 06:49:57 on dist-test-slave-5l3k"
server_key: "c83d74325dc78592324a4ef67c1eed1b"
server_key_iv: "86e643a4a030a6941b0c0757ce358b84"
server_key_version: "encryptionkey@0"
I20260806 06:49:57.218128 24865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:57.226639 24865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:57.226887 24865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:57.227028 24865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:57.227242 24865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:57.227538 24865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:57.227571 24865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.227591 24865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:57.227638 24865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:57.233055 24865 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:35531
I20260806 06:49:57.233160 24997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:35531 every 8 connection(s)
I20260806 06:49:57.233418 24865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:49:57.238668 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.234140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:56971 (local address 127.6.57.126:39329)
0806 06:49:57.234207 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.234209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.234420 (+ 211us) server_negotiation.cc:407] Connection header received
0806 06:49:57.234964 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.234965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.234976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.235026 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:57.235764 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.236405 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.237447 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.237632 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.237685 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.238388 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:57.238390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:57.238390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:57.238394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:57.238407 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:57.238411 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:57.238453 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.238586 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.238610 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:49:57.238829 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.234124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:39329 (local address 127.6.57.67:56971)
0806 06:49:57.234305 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.234337 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.234878 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.235118 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.235121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.235129 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:57.235641 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.235671 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.236511 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.236512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.237369 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.237371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.237707 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.237708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.237822 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.238264 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:57.238264 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:57.238290 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:57.238310 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:57.238453 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:57.238455 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:57.238457 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.238573 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.238646 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 06:49:57.239393 24998 heartbeater.cc:347] Connected to a master server at 127.6.57.126:39329
I20260806 06:49:57.239490 24998 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:57.239658 24998 heartbeater.cc:511] Master 127.6.57.126:39329 requested a full tablet report, sending...
I20260806 06:49:57.240056 24530 ts_manager.cc:194] Registered new tserver with Master: c91368cb0cf84a3bad3b6bb92a222f7c (127.6.57.67:35531)
I20260806 06:49:57.240428 24530 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.67:56971
I20260806 06:49:57.242213 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 24865
I20260806 06:49:57.242339 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:49:57.242553 6373 external_mini_cluster.cc:1428] Setting key e2175e1877edafb8186064dc5634c731
I20260806 06:49:57.243427 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:49:57.247268 24570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.244805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.6.57.126:39329)
0806 06:49:57.244857 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.244858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.244954 (+ 96us) server_negotiation.cc:407] Connection header received
0806 06:49:57.245005 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.245006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.245015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.245069 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:57.245211 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.245622 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.246359 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.246536 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.246585 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.247091 (+ 506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:57.247092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:57.247092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:57.247096 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:57.247115 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:57.247118 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:57.247156 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.247198 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.247216 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:49:57.248705 24530 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:49:57.250095 24530 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:49:57.256260 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46815 (local address 127.0.0.1:47448)
0806 06:49:57.252343 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.252382 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.252446 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.252723 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.252726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.252728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:57.252784 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.252786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.253797 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.253798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.255019 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.255022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.255980 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.255981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.256057 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.256069 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.256140 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.256186 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":47,"threads_started":1}
I20260806 06:49:57.256313 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47448 (local address 127.6.57.66:46815)
0806 06:49:57.252506 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.252511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.252522 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:49:57.252562 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.252564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.252579 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.252697 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:57.252874 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.253708 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.255111 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.255781 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.255848 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.255943 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.256180 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.256230 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260806 06:49:57.256587 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43931 (local address 127.0.0.1:46774)
0806 06:49:57.252791 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.252799 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.252862 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.253159 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.253160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.253162 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:57.253194 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.253195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.254267 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.254268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.255342 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.255344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.256351 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.256353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.256420 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.256424 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.256483 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.256523 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":58,"threads_started":1}
I20260806 06:49:57.256623 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.6.57.65:43931)
0806 06:49:57.252953 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.252957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.252966 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:57.253004 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.253007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.253020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.253089 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:57.253276 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.254172 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.255438 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.256157 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.256229 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.256313 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.256502 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.256544 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":87,"threads_started":1}
I20260806 06:49:57.257023 24792 tablet_service.cc:1467] Processing CreateTablet for tablet fb43d058fe1942228b5a592ec02b124d (DEFAULT_TABLE table=test-table [id=5af56fdd1fcd48cfb73aea7990824ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:57.257169 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:35531 (local address 127.0.0.1:38870)
0806 06:49:57.252514 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.252526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.252568 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.252763 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.252763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.252765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:57.252794 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.252795 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.254694 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.254695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.255601 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.255604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.256990 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.256991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.257035 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.257038 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.257075 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.257101 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":107,"threads_started":1}
I20260806 06:49:57.257211 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.252254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38870 (local address 127.6.57.67:35531)
0806 06:49:57.252577 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.252581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.252590 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:57.252626 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.252628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.252641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.252709 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:57.253794 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.254608 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.255692 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.256906 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.256980 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.257069 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.257095 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.257138 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":86,"threads_started":1}
I20260806 06:49:57.257411 24792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43d058fe1942228b5a592ec02b124d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:57.259155 24928 tablet_service.cc:1467] Processing CreateTablet for tablet fb43d058fe1942228b5a592ec02b124d (DEFAULT_TABLE table=test-table [id=5af56fdd1fcd48cfb73aea7990824ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:57.259297 24657 tablet_service.cc:1467] Processing CreateTablet for tablet fb43d058fe1942228b5a592ec02b124d (DEFAULT_TABLE table=test-table [id=5af56fdd1fcd48cfb73aea7990824ec6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:57.259512 24928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43d058fe1942228b5a592ec02b124d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:57.259649 24657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43d058fe1942228b5a592ec02b124d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:57.260684 25012 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: Bootstrap starting.
I20260806 06:49:57.261445 25012 tablet_bootstrap.cc:654] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:57.261783 25012 log.cc:824] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:57.262568 25014 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Bootstrap starting.
I20260806 06:49:57.262616 25012 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: No bootstrap required, opened a new log
I20260806 06:49:57.262679 25012 ts_tablet_manager.cc:1397] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:57.262986 25015 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: Bootstrap starting.
I20260806 06:49:57.263368 25014 tablet_bootstrap.cc:654] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:57.263659 25014 log.cc:824] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:57.263679 25015 tablet_bootstrap.cc:654] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:57.263972 25015 log.cc:824] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:57.264395 25014 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: No bootstrap required, opened a new log
I20260806 06:49:57.264465 25014 ts_tablet_manager.cc:1397] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:57.264775 25015 tablet_bootstrap.cc:492] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: No bootstrap required, opened a new log
I20260806 06:49:57.264844 25015 ts_tablet_manager.cc:1397] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:57.265376 25012 raft_consensus.cc:348] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.265571 25012 raft_consensus.cc:374] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:57.265610 25012 raft_consensus.cc:729] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a82423cf49f941bfa1045ef6c11503c3, State: Initialized, Role: FOLLOWER
I20260806 06:49:57.265738 25012 consensus_queue.cc:260] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.266006 25012 ts_tablet_manager.cc:1428] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:49:57.266161 24862 heartbeater.cc:503] Master 127.6.57.126:39329 was elected leader, sending a full tablet report...
I20260806 06:49:57.266727 25014 raft_consensus.cc:348] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.266878 25014 raft_consensus.cc:374] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:57.266923 25014 raft_consensus.cc:729] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c91368cb0cf84a3bad3b6bb92a222f7c, State: Initialized, Role: FOLLOWER
I20260806 06:49:57.267053 25014 consensus_queue.cc:260] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.267320 25014 ts_tablet_manager.cc:1428] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 06:49:57.267350 24998 heartbeater.cc:503] Master 127.6.57.126:39329 was elected leader, sending a full tablet report...
I20260806 06:49:57.267280 25015 raft_consensus.cc:348] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.267431 25015 raft_consensus.cc:374] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:57.267473 25015 raft_consensus.cc:729] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3328888665403283d51925209f181f, State: Initialized, Role: FOLLOWER
I20260806 06:49:57.267575 25015 consensus_queue.cc:260] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.267851 25015 ts_tablet_manager.cc:1428] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:49:57.268069 24724 heartbeater.cc:503] Master 127.6.57.126:39329 was elected leader, sending a full tablet report...
I20260806 06:49:57.300411 25019 raft_consensus.cc:482] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:57.300565 25019 raft_consensus.cc:504] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.300949 25019 leader_election.cc:288] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a82423cf49f941bfa1045ef6c11503c3 (127.6.57.66:46815), 5f3328888665403283d51925209f181f (127.6.57.65:43931)
I20260806 06:49:57.303701 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.301115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43931 (local address 127.6.57.67:60085)
0806 06:49:57.301179 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.301191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.301240 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.301373 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.301374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.301375 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:57.301438 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.301440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.302095 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.302096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.302879 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.302881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.303448 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.303449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.303520 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.303523 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.303592 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.303635 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 06:49:57.303701 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.301204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:60085 (local address 127.6.57.65:43931)
0806 06:49:57.301270 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.301272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.301281 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:57.301306 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.301307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.301318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.301361 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:57.301513 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.301985 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.302973 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.303354 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.303437 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.303448 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.303599 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.303637 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:49:57.303877 25009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.301263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:41665 (local address 127.6.57.66:46815)
0806 06:49:57.301304 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.301307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.301350 (+ 43us) server_negotiation.cc:407] Connection header received
0806 06:49:57.301417 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.301418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.301427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.301502 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:57.301652 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.302091 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.303123 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.303477 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.303552 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.303562 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.303773 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.303812 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:49:57.303885 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.301111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46815 (local address 127.6.57.67:41665)
0806 06:49:57.301297 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.301308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:57.301365 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:57.301505 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:57.301506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:57.301508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:57.301594 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.301596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.302192 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.302194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.303021 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:57.303023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.303572 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.303573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:57.303679 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.303682 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:57.303767 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 06:49:57.303804 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260806 06:49:57.304087 24677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43d058fe1942228b5a592ec02b124d" candidate_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3328888665403283d51925209f181f" is_pre_election: true
I20260806 06:49:57.304162 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43d058fe1942228b5a592ec02b124d" candidate_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a82423cf49f941bfa1045ef6c11503c3" is_pre_election: true
I20260806 06:49:57.304260 24677 raft_consensus.cc:2454] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91368cb0cf84a3bad3b6bb92a222f7c in term 0.
I20260806 06:49:57.304302 24813 raft_consensus.cc:2454] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c91368cb0cf84a3bad3b6bb92a222f7c in term 0.
I20260806 06:49:57.304507 24880 leader_election.cc:302] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3328888665403283d51925209f181f, c91368cb0cf84a3bad3b6bb92a222f7c; no voters:
I20260806 06:49:57.304677 25019 raft_consensus.cc:2781] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:49:57.304741 25019 raft_consensus.cc:482] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:49:57.304775 25019 raft_consensus.cc:3037] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:57.305449 25019 raft_consensus.cc:504] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.305595 25019 leader_election.cc:288] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [CANDIDATE]: Term 1 election: Requested vote from peers a82423cf49f941bfa1045ef6c11503c3 (127.6.57.66:46815), 5f3328888665403283d51925209f181f (127.6.57.65:43931)
I20260806 06:49:57.305737 24813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43d058fe1942228b5a592ec02b124d" candidate_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a82423cf49f941bfa1045ef6c11503c3"
I20260806 06:49:57.305768 24677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43d058fe1942228b5a592ec02b124d" candidate_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3328888665403283d51925209f181f"
I20260806 06:49:57.305819 24813 raft_consensus.cc:3037] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:57.305877 24677 raft_consensus.cc:3037] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:57.306602 24813 raft_consensus.cc:2454] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91368cb0cf84a3bad3b6bb92a222f7c in term 1.
I20260806 06:49:57.306766 24883 leader_election.cc:302] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a82423cf49f941bfa1045ef6c11503c3, c91368cb0cf84a3bad3b6bb92a222f7c; no voters:
I20260806 06:49:57.306825 24677 raft_consensus.cc:2454] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c91368cb0cf84a3bad3b6bb92a222f7c in term 1.
I20260806 06:49:57.306867 25019 raft_consensus.cc:2781] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:57.307034 25019 raft_consensus.cc:686] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 1 LEADER]: Becoming Leader. State: Replica: c91368cb0cf84a3bad3b6bb92a222f7c, State: Running, Role: LEADER
I20260806 06:49:57.307130 25019 consensus_queue.cc:237] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } }
I20260806 06:49:57.307915 24530 catalog_manager.cc:5495] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c reported cstate change: term changed from 0 to 1, leader changed from <none> to c91368cb0cf84a3bad3b6bb92a222f7c (127.6.57.67). New cstate: current_term: 1 leader_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c91368cb0cf84a3bad3b6bb92a222f7c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 } health_report { overall_health: UNKNOWN } } }
I20260806 06:49:57.316811 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.314523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.6.57.67:35531)
0806 06:49:57.314583 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:57.314585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:57.314592 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:49:57.314633 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:57.314634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:57.314648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:57.314700 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:57.314793 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.315219 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.315968 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:57.316217 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:57.316314 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:57.316481 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:57.316582 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:49:57.316679 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:57.316705 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:49:57.316720 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 06:49:57.318691 24813 raft_consensus.cc:1261] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 1 FOLLOWER]: Refusing update from remote peer c91368cb0cf84a3bad3b6bb92a222f7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:57.318691 24677 raft_consensus.cc:1261] T fb43d058fe1942228b5a592ec02b124d P 5f3328888665403283d51925209f181f [term 1 FOLLOWER]: Refusing update from remote peer c91368cb0cf84a3bad3b6bb92a222f7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:57.319114 25022 consensus_queue.cc:1035] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3328888665403283d51925209f181f" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 43931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:57.319298 25019 consensus_queue.cc:1035] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a82423cf49f941bfa1045ef6c11503c3" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:57.322618 25026 mvcc.cc:205] Tried to move back new op lower bound from 7315451893014347776 to 7315451892970762240. Current Snapshot: MvccSnapshot[applied={T|T < 7315451893014347776}]
I20260806 06:49:57.323953 25027 mvcc.cc:205] Tried to move back new op lower bound from 7315451893014347776 to 7315451892970762240. Current Snapshot: MvccSnapshot[applied={T|T < 7315451893014347776 or (T in {7315451893014347776})}]
I20260806 06:49:57.324270 24530 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:57.324378 24530 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:57.325079 24530 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdb5ded5dba748599a1bdec44e6c362a: Sending DeleteTablet for 3 replicas of tablet fb43d058fe1942228b5a592ec02b124d
I20260806 06:49:57.325392 24792 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43d058fe1942228b5a592ec02b124d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:57 UTC) from {username='slave'} at 127.0.0.1:47448
I20260806 06:49:57.325433 24657 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43d058fe1942228b5a592ec02b124d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:57 UTC) from {username='slave'} at 127.0.0.1:46774
I20260806 06:49:57.325603 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24590
I20260806 06:49:57.325682 24928 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43d058fe1942228b5a592ec02b124d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:57 UTC) from {username='slave'} at 127.0.0.1:38870
I20260806 06:49:57.327013 25034 tablet_replica.cc:330] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3: stopping tablet replica
I20260806 06:49:57.327206 25034 raft_consensus.cc:2229] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:57.327368 25034 raft_consensus.cc:2258] T fb43d058fe1942228b5a592ec02b124d P a82423cf49f941bfa1045ef6c11503c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:57.327965 25036 tablet_replica.cc:330] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: stopping tablet replica
I20260806 06:49:57.328080 25036 raft_consensus.cc:2229] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:49:57.328220 25036 raft_consensus.cc:2258] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:57.328745 25036 ts_tablet_manager.cc:1905] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:57.330540 25036 ts_tablet_manager.cc:1918] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:57.330610 25036 log.cc:1196] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/fb43d058fe1942228b5a592ec02b124d
I20260806 06:49:57.330840 25036 ts_tablet_manager.cc:1939] T fb43d058fe1942228b5a592ec02b124d P c91368cb0cf84a3bad3b6bb92a222f7c: Deleting consensus metadata
I20260806 06:49:57.331271 24514 catalog_manager.cc:4841] TS c91368cb0cf84a3bad3b6bb92a222f7c (127.6.57.67:35531): tablet fb43d058fe1942228b5a592ec02b124d (table test-table [id=5af56fdd1fcd48cfb73aea7990824ec6]) successfully deleted
W20260806 06:49:57.334021 24514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:43931 (error 108)
I20260806 06:49:57.334523 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.334361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:43931 (local address 127.0.0.1:46780)
0806 06:49:57.334407 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.334460 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 06:49:57.334632 24514 catalog_manager.cc:4584] TS 5f3328888665403283d51925209f181f (127.6.57.65:43931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb43d058fe1942228b5a592ec02b124d: Network error: Client connection negotiation failed: client connection to 127.6.57.65:43931: connect: Connection refused (error 111)
I20260806 06:49:57.334913 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24727
I20260806 06:49:57.343621 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24865
I20260806 06:49:57.343676 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:57.343530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:46815 (local address 127.0.0.1:47450)
0806 06:49:57.343602 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:57.343644 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.66:46815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 06:49:57.349153 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 24499
2026-08-06T06:49:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (668 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:49:57 dist-test-slave-5l3k krb5kdc[25040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:49:57 dist-test-slave-5l3k krb5kdc[25040](info): set up 2 sockets
Aug 06 06:49:57 dist-test-slave-5l3k krb5kdc[25040](info): commencing operation
krb5kdc: starting...
W20260806 06:49:58.392177 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:49:58.513088 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:40769
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:40769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:58.594784 25054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:58.595034 25054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:58.595069 25054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:58.596232 25054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:58.596307 25054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:58.596331 25054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:58.596352 25054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:58.596379 25054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:58.597919 25054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:40769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:40769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:58.598205 25054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:58.598438 25054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:58.603314 25059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:58.603333 25062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.603550 25054 server_base.cc:1034] running on GCE node
W20260806 06:49:58.603377 25060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.603902 25054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:58.604192 25054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:58.605365 25054 hybrid_clock.cc:648] HybridClock initialized: now 1785998998605354 us; error 39 us; skew 500 ppm
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:58.608029 25054 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:49:58.608474 25054 webserver.cc:460] Webserver started at http://127.6.57.126:40707/ using document root <none> and password file <none>
I20260806 06:49:58.608685 25054 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:58.608726 25054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:58.608814 25054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:58.609814 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "61b3f0f1b30c46748977a45d5a44f60e"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "2b1ec6ff2b41ccd4da9bd35db9141648"
server_key_iv: "667d11a8b00da0d1e4859be03b91787a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.610105 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "61b3f0f1b30c46748977a45d5a44f60e"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "2b1ec6ff2b41ccd4da9bd35db9141648"
server_key_iv: "667d11a8b00da0d1e4859be03b91787a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.611536 25054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:49:58.612232 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:58.612432 25054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:58.612524 25054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "61b3f0f1b30c46748977a45d5a44f60e"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "2b1ec6ff2b41ccd4da9bd35db9141648"
server_key_iv: "667d11a8b00da0d1e4859be03b91787a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.612577 25054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:58.629609 25054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:58.631748 25054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:58.631873 25054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:58.636359 25054 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:40769
I20260806 06:49:58.636412 25124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:40769 every 8 connection(s)
I20260806 06:49:58.636794 25054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:58.637709 25125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:58.638727 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25054
I20260806 06:49:58.638823 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:49:58.639068 6373 external_mini_cluster.cc:1428] Setting key 0134ecd5016be6fef0b1f977933e3c62
I20260806 06:49:58.640735 25125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: Bootstrap starting.
I20260806 06:49:58.641330 25125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:58.641673 25125 log.cc:824] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:58.642424 25125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: No bootstrap required, opened a new log
I20260806 06:49:58.644714 25125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } }
I20260806 06:49:58.644863 25125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:58.644888 25125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b3f0f1b30c46748977a45d5a44f60e, State: Initialized, Role: FOLLOWER
I20260806 06:49:58.645018 25125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } }
I20260806 06:49:58.645097 25125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:58.645120 25125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:58.645183 25125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:58.645710 25125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } }
I20260806 06:49:58.645794 25125 leader_election.cc:302] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61b3f0f1b30c46748977a45d5a44f60e; no voters:
I20260806 06:49:58.645975 25125 leader_election.cc:288] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:49:58.646014 25130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:58.646174 25130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [term 1 LEADER]: Becoming Leader. State: Replica: 61b3f0f1b30c46748977a45d5a44f60e, State: Running, Role: LEADER
I20260806 06:49:58.646263 25125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:58.646256 25130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } }
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:58.646694 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61b3f0f1b30c46748977a45d5a44f60e. Latest consensus state: current_term: 1 leader_uuid: "61b3f0f1b30c46748977a45d5a44f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } } }
I20260806 06:49:58.646772 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:58.646991 25135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:58.647384 25132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61b3f0f1b30c46748977a45d5a44f60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61b3f0f1b30c46748977a45d5a44f60e" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 40769 } } }
I20260806 06:49:58.647493 25132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:58.647759 25135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:58.650489 25135 catalog_manager.cc:1347] Generated new cluster ID: 6c78bc61702e412691e7afeb0a0fdf63
I20260806 06:49:58.650549 25135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:58.651490 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:58.639618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.6.57.126:40769)
0806 06:49:58.639822 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:58.639832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:58.639865 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:49:58.640427 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:58.640437 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:58.640498 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:58.640644 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:58.641358 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.642707 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.643487 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.644114 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.644246 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:58.647198 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:58.647202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:58.647202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:58.647213 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:58.649353 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:58.649755 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:58.649759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:58.649759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:58.649876 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:58.650138 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:58.650139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:58.650140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:58.650429 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:58.650614 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:58.651153 (+ 539us) server_negotiation.cc:299] Negotiation successful
0806 06:49:58.651291 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260806 06:49:58.662833 25135 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:49:58.663324 25135 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:58.666802 25135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: Generated new TSK 0
I20260806 06:49:58.666977 25135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:49:58.699627 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:58.782227 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:58.782420 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:58.782444 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:58.783532 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:58.783568 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:58.783600 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:58.785049 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:58.785228 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:58.785404 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:58.790316 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.790405 25151 server_base.cc:1034] running on GCE node
W20260806 06:49:58.790349 25156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:58.790306 25159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.790769 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:58.791018 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:58.792174 25151 hybrid_clock.cc:648] HybridClock initialized: now 1785998998792156 us; error 43 us; skew 500 ppm
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:58.794643 25151 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:49:58.795038 25151 webserver.cc:460] Webserver started at http://127.6.57.65:45329/ using document root <none> and password file <none>
I20260806 06:49:58.795269 25151 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:58.795333 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:58.795480 25151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:58.796468 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "4f729692f5e44c3c8ca1e9da4517c913"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "d611b02627451b0caaad38a49044a158"
server_key_iv: "ab9200996f692c2a092176ac782e0a6a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.796836 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "4f729692f5e44c3c8ca1e9da4517c913"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "d611b02627451b0caaad38a49044a158"
server_key_iv: "ab9200996f692c2a092176ac782e0a6a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.798148 25151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:49:58.798940 25166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:58.799166 25151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:49:58.799299 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "4f729692f5e44c3c8ca1e9da4517c913"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "d611b02627451b0caaad38a49044a158"
server_key_iv: "ab9200996f692c2a092176ac782e0a6a"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.799376 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:58.844589 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:58.846661 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:58.846825 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:58.847103 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:58.847504 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:58.847541 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:58.847601 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:58.847626 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:58.853590 25151 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:41429
I20260806 06:49:58.853659 25282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:41429 every 8 connection(s)
I20260806 06:49:58.854076 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:58.854684 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25151
I20260806 06:49:58.854797 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:49:58.855046 6373 external_mini_cluster.cc:1428] Setting key fc3b9a0c0d6f31268087128eba6e8b72
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
I20260806 06:49:58.865483 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:58.855194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:38269 (local address 127.6.57.126:40769)
0806 06:49:58.855268 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:58.855270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:58.855281 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:49:58.855889 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:58.855890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:58.855902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:58.855951 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:58.857233 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.858071 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.859576 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.859871 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.859927 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:58.862460 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:58.862462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:58.862462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:58.862470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:58.864060 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:58.864463 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:58.864464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:58.864465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:58.864528 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:58.864779 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:58.864780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:58.864780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:58.864910 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:58.864981 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:58.865260 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 06:49:58.865376 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:49:58.865541 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:58.854851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:40769 (local address 127.6.57.65:38269)
0806 06:49:58.855040 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:58.855098 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:58.855791 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:58.856091 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:58.856096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:58.856883 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:58.857094 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:58.857099 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.858196 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.858197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:58.859470 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:58.859473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:58.860007 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:58.860008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:58.860138 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:58.860673 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:58.860681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:58.862365 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:58.864147 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:58.864154 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:58.864156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:58.864380 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:58.864613 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:58.864613 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:58.864613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:58.864708 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:58.864975 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:58.864979 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:58.865127 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 06:49:58.865229 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 06:49:58.865351 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260806 06:49:58.866065 25283 heartbeater.cc:347] Connected to a master server at 127.6.57.126:40769
I20260806 06:49:58.866199 25283 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:58.866375 25283 heartbeater.cc:511] Master 127.6.57.126:40769 requested a full tablet report, sending...
I20260806 06:49:58.866936 25087 ts_manager.cc:194] Registered new tserver with Master: 4f729692f5e44c3c8ca1e9da4517c913 (127.6.57.65:41429)
I20260806 06:49:58.867538 25087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:38269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:49:58.882251 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:58.960747 25288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:58.960916 25288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:58.960937 25288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:58.961942 25288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:58.961974 25288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:58.961998 25288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:58.963387 25288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:58.963567 25288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:58.963728 25288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:58.968627 25294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:58.968628 25293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.968835 25288 server_base.cc:1034] running on GCE node
W20260806 06:49:58.968613 25296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:58.969089 25288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:58.969295 25288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:58.970513 25288 hybrid_clock.cc:648] HybridClock initialized: now 1785998998970464 us; error 27 us; skew 500 ppm
Aug 06 06:49:58 dist-test-slave-5l3k krb5kdc[25040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:58.973062 25288 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:49:58.973558 25288 webserver.cc:460] Webserver started at http://127.6.57.66:38041/ using document root <none> and password file <none>
I20260806 06:49:58.973788 25288 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:58.973855 25288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:58.974006 25288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:58.975190 25288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "1176e310e7cc4f6a93d57503782fd480"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "c3cffac03f2eeaa3f12a8d40f474051d"
server_key_iv: "fc2354dfae507c4641e4480743394d21"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.975566 25288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "1176e310e7cc4f6a93d57503782fd480"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "c3cffac03f2eeaa3f12a8d40f474051d"
server_key_iv: "fc2354dfae507c4641e4480743394d21"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.976879 25288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:49:58.977600 25303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:58.977785 25288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:49:58.977886 25288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "1176e310e7cc4f6a93d57503782fd480"
format_stamp: "Formatted at 2026-08-06 06:49:58 on dist-test-slave-5l3k"
server_key: "c3cffac03f2eeaa3f12a8d40f474051d"
server_key_iv: "fc2354dfae507c4641e4480743394d21"
server_key_version: "encryptionkey@0"
I20260806 06:49:58.977985 25288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:58.998413 25288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.000285 25288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.000455 25288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.000685 25288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:59.001084 25288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:59.001147 25288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.001181 25288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:59.001209 25288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.007493 25288 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:36561
I20260806 06:49:59.007553 25420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:36561 every 8 connection(s)
I20260806 06:49:59.007856 25288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
Aug 06 06:49:59 dist-test-slave-5l3k krb5kdc[25040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998998, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:59.017098 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25288
I20260806 06:49:59.017210 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:49:59.017346 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.008569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:50583 (local address 127.6.57.126:40769)
0806 06:49:59.008653 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.008656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.008846 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:49:59.009351 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.009352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.009364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.009415 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.010329 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.010891 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.011835 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.012029 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.012099 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.014199 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.014200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.014200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:59.014207 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.015795 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.016240 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.016241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.016242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.016304 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.016568 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.016569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.016570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.016686 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.016758 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.017086 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.017200 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:49:59.017447 6373 external_mini_cluster.cc:1428] Setting key e9e5d0ea1504c089db00a76ade5e2f37
I20260806 06:49:59.017500 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.008541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:40769 (local address 127.6.57.66:50583)
0806 06:49:59.008720 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.008754 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.009250 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.009497 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.009500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.010047 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:59.010229 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.010234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.010986 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.010988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.011738 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.011740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.012106 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.012107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.012225 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.012680 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:59.012688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:59.014114 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:59.015887 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:59.015894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:59.015896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:59.016160 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:59.016373 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:59.016374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:59.016374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:59.016487 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:59.016762 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:59.016769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:59.016931 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.017066 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.017261 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260806 06:49:59.018064 25421 heartbeater.cc:347] Connected to a master server at 127.6.57.126:40769
I20260806 06:49:59.018190 25421 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:59.018361 25421 heartbeater.cc:511] Master 127.6.57.126:40769 requested a full tablet report, sending...
I20260806 06:49:59.018776 25087 ts_manager.cc:194] Registered new tserver with Master: 1176e310e7cc4f6a93d57503782fd480 (127.6.57.66:36561)
I20260806 06:49:59.019166 25087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:50583
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:49:59.042833 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:37833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:49:59.124696 25426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:59.124879 25426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:59.124902 25426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:59.126025 25426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:59.126068 25426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:59.126093 25426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:59.127619 25426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:37833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:40769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:59.127883 25426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:59.128120 25426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:59.133066 25435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.133116 25432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.133215 25431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.133416 25426 server_base.cc:1034] running on GCE node
I20260806 06:49:59.133587 25426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:59.133759 25426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:59.134889 25426 hybrid_clock.cc:648] HybridClock initialized: now 1785998999134876 us; error 26 us; skew 500 ppm
Aug 06 06:49:59 dist-test-slave-5l3k krb5kdc[25040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:49:59.137270 25426 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:49:59.137712 25426 webserver.cc:460] Webserver started at http://127.6.57.67:38007/ using document root <none> and password file <none>
I20260806 06:49:59.137964 25426 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:59.138008 25426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:59.138180 25426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:59.139227 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "5d57d164f89f42bf9b6c66cf2705bf8c"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "1b2448e73958cc5fba12dcd574346c5b"
server_key_iv: "a7876b771db09ef4106fc91107a6b1e9"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.139554 25426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "5d57d164f89f42bf9b6c66cf2705bf8c"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "1b2448e73958cc5fba12dcd574346c5b"
server_key_iv: "a7876b771db09ef4106fc91107a6b1e9"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.140815 25426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:49:59.141539 25443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.141798 25426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:49:59.141872 25426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "5d57d164f89f42bf9b6c66cf2705bf8c"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "1b2448e73958cc5fba12dcd574346c5b"
server_key_iv: "a7876b771db09ef4106fc91107a6b1e9"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.141948 25426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:59.150575 25426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.152372 25426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.152504 25426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.152726 25426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:59.153033 25426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:59.153064 25426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.153101 25426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:59.153152 25426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.158871 25426 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:33969
I20260806 06:49:59.158972 25560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:33969 every 8 connection(s)
I20260806 06:49:59.159240 25426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:49:59 dist-test-slave-5l3k krb5kdc[25040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785998999, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:49:59.167268 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.159921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:58055 (local address 127.6.57.126:40769)
0806 06:49:59.159967 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.159969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.160182 (+ 213us) server_negotiation.cc:407] Connection header received
0806 06:49:59.160600 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.160601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.160613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.160662 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.161452 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.161891 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.162846 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.163013 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.163061 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.164822 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.164824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.164824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:59.164831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.165993 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.166360 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.166360 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.166361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.166410 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.166699 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.166700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.166701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.166826 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.166914 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.167144 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.167205 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:49:59.167479 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25426
I20260806 06:49:59.167531 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.159905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:40769 (local address 127.6.57.67:58055)
0806 06:49:59.160071 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.160103 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.160513 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.160707 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.160710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.161217 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:59.161386 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.161390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.161958 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.161959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.162780 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.162783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.163077 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.163077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.163159 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.163512 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:49:59.163519 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:59.164741 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:59.166072 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:59.166078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:59.166080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:59.166296 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:59.166467 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:49:59.166468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:49:59.166468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:49:59.166621 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:49:59.166914 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:59.166920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:59.167027 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.167115 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.167289 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260806 06:49:59.167572 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:49:59.167778 6373 external_mini_cluster.cc:1428] Setting key 310e62cd1372e6759038f6ff5e1e4671
I20260806 06:49:59.168121 25561 heartbeater.cc:347] Connected to a master server at 127.6.57.126:40769
I20260806 06:49:59.168241 25561 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:59.168494 25561 heartbeater.cc:511] Master 127.6.57.126:40769 requested a full tablet report, sending...
I20260806 06:49:59.168937 25087 ts_manager.cc:194] Registered new tserver with Master: 5d57d164f89f42bf9b6c66cf2705bf8c (127.6.57.67:33969)
I20260806 06:49:59.169308 25087 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:58055
I20260806 06:49:59.179554 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:49:59.188884 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.181805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.6.57.126:40769)
0806 06:49:59.181857 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.181859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.181968 (+ 109us) server_negotiation.cc:407] Connection header received
0806 06:49:59.182063 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.182064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.182073 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.182123 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.182774 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.183255 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.184002 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.184201 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.184273 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.185627 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.185629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.185629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:49:59.185637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.187175 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.187742 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.187743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.187744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.187837 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:49:59.188128 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:49:59.188129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:49:59.188129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:49:59.188345 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.188470 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.188667 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.188758 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 06:49:59.192101 25087 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:49:59.193619 25087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:49:59.199928 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.195931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41429 (local address 127.0.0.1:60832)
0806 06:49:59.196095 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.196142 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.196211 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.196446 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.196450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.196452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.196523 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.196524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.197567 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.197568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.198705 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.198708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.199663 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.199743 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.199754 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.199813 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.199858 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
I20260806 06:49:59.200052 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.195959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:36561 (local address 127.0.0.1:36426)
0806 06:49:59.196660 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.196667 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.196729 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.196970 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.196971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.196973 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.197020 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.197021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.198087 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.198089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.199184 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.199186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.199817 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.199879 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.199885 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.199942 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.199983 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":13,"thread_start_us":76,"threads_started":1}
I20260806 06:49:59.200067 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.196219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.6.57.66:36561)
0806 06:49:59.196388 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.196391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.196736 (+ 345us) server_negotiation.cc:407] Connection header received
0806 06:49:59.196791 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.196793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.196809 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.196882 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.197092 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.197980 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199272 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.199731 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199815 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.199870 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.199957 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.199996 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260806 06:49:59.200487 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.196036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.6.57.65:41429)
0806 06:49:59.196274 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.196279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.196289 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:49:59.196325 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.196326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.196341 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.196428 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.196617 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.197467 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.198796 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.199498 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199577 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.199630 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.200390 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.200432 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
I20260806 06:49:59.200897 25352 tablet_service.cc:1467] Processing CreateTablet for tablet 571d9296e3d241c18dcba71e6846ab11 (DEFAULT_TABLE table=test-table [id=3a3e926bb60a4bdc8c34f6e616d2f6b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.201107 25214 tablet_service.cc:1467] Processing CreateTablet for tablet 571d9296e3d241c18dcba71e6846ab11 (DEFAULT_TABLE table=test-table [id=3a3e926bb60a4bdc8c34f6e616d2f6b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.201243 25352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571d9296e3d241c18dcba71e6846ab11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.201471 25214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571d9296e3d241c18dcba71e6846ab11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.203510 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.195958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:33969 (local address 127.0.0.1:53658)
0806 06:49:59.196525 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.196536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.196590 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.197706 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.197707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.197710 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.197816 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.197818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.199042 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.199043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.200261 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.200264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.203265 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.203266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.203329 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.203333 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.203402 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.203443 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":43,"thread_start_us":92,"threads_started":1}
I20260806 06:49:59.203533 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.196187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.6.57.67:33969)
0806 06:49:59.196448 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.196450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.197558 (+ 1108us) server_negotiation.cc:407] Connection header received
0806 06:49:59.197597 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.197599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.197612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.197694 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.197893 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.198943 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.200354 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.203177 (+ 2823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.203252 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.203308 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.203416 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.203459 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260806 06:49:59.204272 25491 tablet_service.cc:1467] Processing CreateTablet for tablet 571d9296e3d241c18dcba71e6846ab11 (DEFAULT_TABLE table=test-table [id=3a3e926bb60a4bdc8c34f6e616d2f6b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.204497 25578 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: Bootstrap starting.
I20260806 06:49:59.204644 25491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 571d9296e3d241c18dcba71e6846ab11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.205103 25578 tablet_bootstrap.cc:654] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.205626 25578 log.cc:824] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.206377 25578 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: No bootstrap required, opened a new log
I20260806 06:49:59.206430 25578 ts_tablet_manager.cc:1397] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:59.206808 25579 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Bootstrap starting.
I20260806 06:49:59.207469 25579 tablet_bootstrap.cc:654] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.207862 25581 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Bootstrap starting.
I20260806 06:49:59.208052 25579 log.cc:824] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.208173 25578 raft_consensus.cc:348] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.208364 25578 raft_consensus.cc:374] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.208417 25578 raft_consensus.cc:729] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f729692f5e44c3c8ca1e9da4517c913, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.208554 25581 tablet_bootstrap.cc:654] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.208567 25578 consensus_queue.cc:260] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.208915 25578 ts_tablet_manager.cc:1428] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 06:49:59.208940 25283 heartbeater.cc:503] Master 127.6.57.126:40769 was elected leader, sending a full tablet report...
I20260806 06:49:59.209141 25581 log.cc:824] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.209395 25579 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: No bootstrap required, opened a new log
I20260806 06:49:59.209456 25579 ts_tablet_manager.cc:1397] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 06:49:59.210213 25581 tablet_bootstrap.cc:492] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: No bootstrap required, opened a new log
I20260806 06:49:59.210282 25581 ts_tablet_manager.cc:1397] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:49:59.211427 25579 raft_consensus.cc:348] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.211615 25579 raft_consensus.cc:374] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.211653 25579 raft_consensus.cc:729] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1176e310e7cc4f6a93d57503782fd480, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.211786 25579 consensus_queue.cc:260] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.212055 25579 ts_tablet_manager.cc:1428] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:49:59.212098 25421 heartbeater.cc:503] Master 127.6.57.126:40769 was elected leader, sending a full tablet report...
I20260806 06:49:59.212170 25581 raft_consensus.cc:348] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.212321 25581 raft_consensus.cc:374] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.212359 25581 raft_consensus.cc:729] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d57d164f89f42bf9b6c66cf2705bf8c, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.212492 25581 consensus_queue.cc:260] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.212790 25581 ts_tablet_manager.cc:1428] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 06:49:59.212798 25561 heartbeater.cc:503] Master 127.6.57.126:40769 was elected leader, sending a full tablet report...
I20260806 06:49:59.239778 25587 raft_consensus.cc:482] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:59.239888 25587 raft_consensus.cc:504] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.240235 25587 leader_election.cc:288] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f729692f5e44c3c8ca1e9da4517c913 (127.6.57.65:41429), 1176e310e7cc4f6a93d57503782fd480 (127.6.57.66:36561)
I20260806 06:49:59.244012 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.240543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:36095 (local address 127.6.57.66:36561)
0806 06:49:59.240594 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.240595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.240603 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:49:59.240640 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.240641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.240649 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.240709 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.240913 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.241654 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243096 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.243676 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243745 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.243754 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.243910 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.243948 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:49:59.244012 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.240506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:47773 (local address 127.6.57.65:41429)
0806 06:49:59.240553 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.240555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.240686 (+ 131us) server_negotiation.cc:407] Connection header received
0806 06:49:59.240749 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.240750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.240763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.240837 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.241036 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.241714 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243095 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.243668 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243745 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.243755 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.243911 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.243948 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 06:49:59.244028 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.240399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:36561 (local address 127.6.57.67:36095)
0806 06:49:59.240458 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.240469 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.240513 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.240726 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.240727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.240729 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.240841 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.240844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.241731 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.241732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.242995 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.242998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.243756 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.243832 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.243834 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.243888 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.243934 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":68}
I20260806 06:49:59.244040 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.240430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41429 (local address 127.6.57.67:47773)
0806 06:49:59.240590 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.240600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.240693 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.240841 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.240842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.240844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.240915 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.240917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.241791 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.241791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.242996 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.242998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.243757 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.243757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.243828 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.243832 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.243888 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.243934 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":12,"thread_start_us":91,"threads_started":1}
I20260806 06:49:59.244406 25373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571d9296e3d241c18dcba71e6846ab11" candidate_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1176e310e7cc4f6a93d57503782fd480" is_pre_election: true
I20260806 06:49:59.244553 25373 raft_consensus.cc:2454] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d57d164f89f42bf9b6c66cf2705bf8c in term 0.
I20260806 06:49:59.244535 25235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571d9296e3d241c18dcba71e6846ab11" candidate_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f729692f5e44c3c8ca1e9da4517c913" is_pre_election: true
I20260806 06:49:59.244692 25235 raft_consensus.cc:2454] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d57d164f89f42bf9b6c66cf2705bf8c in term 0.
I20260806 06:49:59.244841 25446 leader_election.cc:302] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1176e310e7cc4f6a93d57503782fd480, 5d57d164f89f42bf9b6c66cf2705bf8c; no voters:
I20260806 06:49:59.244982 25587 raft_consensus.cc:2781] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:49:59.245065 25587 raft_consensus.cc:482] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:49:59.245101 25587 raft_consensus.cc:3037] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.245779 25587 raft_consensus.cc:504] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.245919 25587 leader_election.cc:288] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [CANDIDATE]: Term 1 election: Requested vote from peers 4f729692f5e44c3c8ca1e9da4517c913 (127.6.57.65:41429), 1176e310e7cc4f6a93d57503782fd480 (127.6.57.66:36561)
I20260806 06:49:59.246086 25373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571d9296e3d241c18dcba71e6846ab11" candidate_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1176e310e7cc4f6a93d57503782fd480"
I20260806 06:49:59.246093 25235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "571d9296e3d241c18dcba71e6846ab11" candidate_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f729692f5e44c3c8ca1e9da4517c913"
I20260806 06:49:59.246156 25373 raft_consensus.cc:3037] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.246192 25235 raft_consensus.cc:3037] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.246994 25373 raft_consensus.cc:2454] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d57d164f89f42bf9b6c66cf2705bf8c in term 1.
I20260806 06:49:59.247102 25235 raft_consensus.cc:2454] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d57d164f89f42bf9b6c66cf2705bf8c in term 1.
I20260806 06:49:59.247179 25446 leader_election.cc:302] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1176e310e7cc4f6a93d57503782fd480, 5d57d164f89f42bf9b6c66cf2705bf8c; no voters:
I20260806 06:49:59.247303 25587 raft_consensus.cc:2781] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:59.247496 25587 raft_consensus.cc:686] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 1 LEADER]: Becoming Leader. State: Replica: 5d57d164f89f42bf9b6c66cf2705bf8c, State: Running, Role: LEADER
I20260806 06:49:59.247601 25587 consensus_queue.cc:237] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } }
I20260806 06:49:59.248420 25087 catalog_manager.cc:5495] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d57d164f89f42bf9b6c66cf2705bf8c (127.6.57.67). New cstate: current_term: 1 leader_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d57d164f89f42bf9b6c66cf2705bf8c" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 33969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 } health_report { overall_health: UNKNOWN } } }
I20260806 06:49:59.257010 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.254984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53664 (local address 127.6.57.67:33969)
0806 06:49:59.255028 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.255031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.255041 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:49:59.255129 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.255129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.255143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.255196 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:59.255288 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.255691 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.256389 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.256555 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.256602 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.256728 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:59.256803 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:49:59.256868 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.256909 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.256916 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 06:49:59.258364 25422 tablet.cc:2369] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:49:59.258590 25235 raft_consensus.cc:1261] T 571d9296e3d241c18dcba71e6846ab11 P 4f729692f5e44c3c8ca1e9da4517c913 [term 1 FOLLOWER]: Refusing update from remote peer 5d57d164f89f42bf9b6c66cf2705bf8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:59.258590 25373 raft_consensus.cc:1261] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 1 FOLLOWER]: Refusing update from remote peer 5d57d164f89f42bf9b6c66cf2705bf8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:59.258947 25589 consensus_queue.cc:1035] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1176e310e7cc4f6a93d57503782fd480" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 36561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:59.259083 25587 consensus_queue.cc:1035] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f729692f5e44c3c8ca1e9da4517c913" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 41429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:59.263655 25085 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:59.263738 25085 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:59.264436 25085 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61b3f0f1b30c46748977a45d5a44f60e: Sending DeleteTablet for 3 replicas of tablet 571d9296e3d241c18dcba71e6846ab11
I20260806 06:49:59.264758 25352 tablet_service.cc:1514] Processing DeleteTablet for tablet 571d9296e3d241c18dcba71e6846ab11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:36426
I20260806 06:49:59.264817 25491 tablet_service.cc:1514] Processing DeleteTablet for tablet 571d9296e3d241c18dcba71e6846ab11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53658
I20260806 06:49:59.265033 25603 tablet_replica.cc:330] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: stopping tablet replica
I20260806 06:49:59.265065 25214 tablet_service.cc:1514] Processing DeleteTablet for tablet 571d9296e3d241c18dcba71e6846ab11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:60832
I20260806 06:49:59.265164 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25151
I20260806 06:49:59.265795 25603 raft_consensus.cc:2229] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:49:59.265992 25603 raft_consensus.cc:2258] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:59.267098 25602 tablet_replica.cc:330] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: stopping tablet replica
I20260806 06:49:59.267225 25602 raft_consensus.cc:2229] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:59.267323 25602 raft_consensus.cc:2258] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:59.267714 25602 ts_tablet_manager.cc:1905] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:59.267746 25603 ts_tablet_manager.cc:1905] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:59.269179 25602 ts_tablet_manager.cc:1918] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:59.269263 25602 log.cc:1196] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/571d9296e3d241c18dcba71e6846ab11
I20260806 06:49:59.269529 25602 ts_tablet_manager.cc:1939] T 571d9296e3d241c18dcba71e6846ab11 P 1176e310e7cc4f6a93d57503782fd480: Deleting consensus metadata
I20260806 06:49:59.269591 25603 ts_tablet_manager.cc:1918] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:59.269671 25603 log.cc:1196] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/571d9296e3d241c18dcba71e6846ab11
I20260806 06:49:59.269948 25603 ts_tablet_manager.cc:1939] T 571d9296e3d241c18dcba71e6846ab11 P 5d57d164f89f42bf9b6c66cf2705bf8c: Deleting consensus metadata
I20260806 06:49:59.269948 25072 catalog_manager.cc:4841] TS 1176e310e7cc4f6a93d57503782fd480 (127.6.57.66:36561): tablet 571d9296e3d241c18dcba71e6846ab11 (table test-table [id=3a3e926bb60a4bdc8c34f6e616d2f6b2]) successfully deleted
I20260806 06:49:59.270319 25073 catalog_manager.cc:4841] TS 5d57d164f89f42bf9b6c66cf2705bf8c (127.6.57.67:33969): tablet 571d9296e3d241c18dcba71e6846ab11 (table test-table [id=3a3e926bb60a4bdc8c34f6e616d2f6b2]) successfully deleted
W20260806 06:49:59.272053 25071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:41429 (error 108)
I20260806 06:49:59.272539 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.272414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:41429 (local address 127.0.0.1:60846)
0806 06:49:59.272465 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.272506 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:41429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 06:49:59.272629 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25288
W20260806 06:49:59.272727 25071 catalog_manager.cc:4584] TS 4f729692f5e44c3c8ca1e9da4517c913 (127.6.57.65:41429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 571d9296e3d241c18dcba71e6846ab11: Network error: Client connection negotiation failed: client connection to 127.6.57.65:41429: connect: Connection refused (error 111)
I20260806 06:49:59.279531 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25426
I20260806 06:49:59.285156 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25054
2026-08-06T06:49:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1936 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T06:49:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:49:59Z Disabled control of system clock
I20260806 06:49:59.313481 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46565
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:46565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:49:59.391853 25609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:59.392033 25609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:59.392051 25609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:59.393179 25609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:49:59.393219 25609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:59.393231 25609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:59.393267 25609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:49:59.393296 25609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:49:59.394646 25609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:46565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:59.394881 25609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:59.395172 25609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:59.399960 25615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.400010 25617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.399973 25614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.400255 25609 server_base.cc:1034] running on GCE node
I20260806 06:49:59.400477 25609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:59.400710 25609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:59.401888 25609 hybrid_clock.cc:648] HybridClock initialized: now 1785998999401884 us; error 39 us; skew 500 ppm
I20260806 06:49:59.403147 25609 webserver.cc:460] Webserver started at http://127.6.57.126:40549/ using document root <none> and password file <none>
I20260806 06:49:59.403393 25609 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:59.403461 25609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:59.403632 25609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:59.404587 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "3d7187704fac4221921cd05e786e9ccf"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "2f9931f54039a495e0b747a6f9a85090"
server_key_iv: "937188f5d633837f7d416ea38543193e"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.404907 25609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "3d7187704fac4221921cd05e786e9ccf"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "2f9931f54039a495e0b747a6f9a85090"
server_key_iv: "937188f5d633837f7d416ea38543193e"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.406163 25609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:49:59.406905 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.407078 25609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.407192 25609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "3d7187704fac4221921cd05e786e9ccf"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "2f9931f54039a495e0b747a6f9a85090"
server_key_iv: "937188f5d633837f7d416ea38543193e"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.407267 25609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:59.421144 25609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.421375 25609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.421466 25609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.425304 25609 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:46565
I20260806 06:49:59.425356 25675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:46565 every 8 connection(s)
I20260806 06:49:59.425666 25609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:49:59.426657 25676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.427534 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25609
I20260806 06:49:59.427619 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:49:59.427799 6373 external_mini_cluster.cc:1428] Setting key 05b31bdf6a138ebfca9d6d8cd3827aba
I20260806 06:49:59.429401 25676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: Bootstrap starting.
I20260806 06:49:59.430095 25676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.430399 25676 log.cc:824] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.431130 25676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: No bootstrap required, opened a new log
I20260806 06:49:59.433482 25676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } }
I20260806 06:49:59.433641 25676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.433703 25676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7187704fac4221921cd05e786e9ccf, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.433854 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.428320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48688 (local address 127.6.57.126:46565)
0806 06:49:59.428550 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.428558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.428590 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:49:59.429105 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.429114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.429193 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.429339 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.429557 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.430807 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.431667 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.432270 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.432405 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.433036 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.433041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.433041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:59.433269 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.433306 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:59.433319 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.433406 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.433495 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.433574 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260806 06:49:59.433831 25676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } }
I20260806 06:49:59.433900 25676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:49:59.433929 25676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:49:59.433972 25676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.434623 25676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } }
I20260806 06:49:59.434728 25676 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d7187704fac4221921cd05e786e9ccf; no voters:
I20260806 06:49:59.434864 25676 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:49:59.434937 25681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:59.435075 25681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [term 1 LEADER]: Becoming Leader. State: Replica: 3d7187704fac4221921cd05e786e9ccf, State: Running, Role: LEADER
I20260806 06:49:59.435133 25676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:49:59.435154 25681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } }
I20260806 06:49:59.435451 25682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d7187704fac4221921cd05e786e9ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } } }
I20260806 06:49:59.435593 25682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [sys.catalog]: This master's current role is: LEADER
I20260806 06:49:59.435459 25683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7187704fac4221921cd05e786e9ccf. Latest consensus state: current_term: 1 leader_uuid: "3d7187704fac4221921cd05e786e9ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d7187704fac4221921cd05e786e9ccf" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 46565 } } }
I20260806 06:49:59.435698 25683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf [sys.catalog]: This master's current role is: LEADER
W20260806 06:49:59.436914 25698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:49:59.437001 25698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:49:59.437160 25690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:49:59.437352 25690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:49:59.438794 25690 catalog_manager.cc:1347] Generated new cluster ID: 571e0b97095647daa2f09cb96ccc913c
I20260806 06:49:59.438874 25690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:49:59.458680 25690 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:49:59.459180 25690 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:49:59.470762 25690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: Generated new TSK 0
I20260806 06:49:59.470943 25690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:49:59.473783 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--builtin_ntp_servers=127.6.57.84:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:49:59.569747 25702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:59.569939 25702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:59.569960 25702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:59.571135 25702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:59.571174 25702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:59.571198 25702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:49:59.572541 25702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:59.572770 25702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:59.573086 25702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:59.578014 25709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.578019 25707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.578068 25711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.578298 25702 server_base.cc:1034] running on GCE node
I20260806 06:49:59.578589 25702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:59.578783 25702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:59.579931 25702 hybrid_clock.cc:648] HybridClock initialized: now 1785998999579921 us; error 31 us; skew 500 ppm
I20260806 06:49:59.581130 25702 webserver.cc:460] Webserver started at http://127.6.57.65:39861/ using document root <none> and password file <none>
I20260806 06:49:59.581377 25702 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:59.581422 25702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:59.581583 25702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:59.582703 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "7d9ce95a6ecf16a43787f2825bec3880"
server_key_iv: "f677d5ad25cf3d6e7b56df831590322d"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.583071 25702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "7d9ce95a6ecf16a43787f2825bec3880"
server_key_iv: "f677d5ad25cf3d6e7b56df831590322d"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.584424 25702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:49:59.585151 25717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.585330 25702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:49:59.585429 25702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "7d9ce95a6ecf16a43787f2825bec3880"
server_key_iv: "f677d5ad25cf3d6e7b56df831590322d"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.585511 25702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:59.608395 25702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.608713 25702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.608876 25702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.609107 25702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:59.609452 25702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:59.609506 25702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.609551 25702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:59.609577 25702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.615182 25702 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:46563
I20260806 06:49:59.615257 25835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:46563 every 8 connection(s)
I20260806 06:49:59.615579 25702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:49:59.620627 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.616296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:34381 (local address 127.6.57.126:46565)
0806 06:49:59.616340 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.616342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.616604 (+ 262us) server_negotiation.cc:407] Connection header received
0806 06:49:59.617051 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.617051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.617061 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.617124 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.617793 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.618290 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.619365 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.619548 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.619598 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.620373 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.620374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.620375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:59.620379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.620391 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:59.620395 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.620436 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.620549 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.620572 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:49:59.620842 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.616278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46565 (local address 127.6.57.65:34381)
0806 06:49:59.616465 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.616504 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.616971 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.617176 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.617179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.617188 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:59.617695 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.617699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.618406 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.618407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.619289 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.619292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.619629 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.619630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.619736 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.620241 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:59.620241 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:59.620268 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:59.620291 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:59.620468 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:59.620473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:59.620476 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.620535 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.620623 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260806 06:49:59.620920 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25702
I20260806 06:49:59.621008 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:49:59.621192 6373 external_mini_cluster.cc:1428] Setting key 57b6c37044e53c8e1dadd8a871c612aa
I20260806 06:49:59.621989 25836 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46565
I20260806 06:49:59.622102 25836 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:59.622255 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--builtin_ntp_servers=127.6.57.84:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:49:59.622283 25836 heartbeater.cc:511] Master 127.6.57.126:46565 requested a full tablet report, sending...
I20260806 06:49:59.622772 25639 ts_manager.cc:194] Registered new tserver with Master: 66c08f8cb3b845fbae34ef5d04bcdfb9 (127.6.57.65:46563)
I20260806 06:49:59.623333 25639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.65:34381
W20260806 06:49:59.706180 25839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:59.706367 25839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:59.706388 25839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:59.707617 25839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:59.707659 25839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:59.707683 25839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:49:59.709028 25839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:59.709209 25839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:59.709506 25839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:59.714514 25845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.714529 25848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.714581 25846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.714665 25839 server_base.cc:1034] running on GCE node
I20260806 06:49:59.714963 25839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:59.715237 25839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:59.716405 25839 hybrid_clock.cc:648] HybridClock initialized: now 1785998999716397 us; error 51 us; skew 500 ppm
I20260806 06:49:59.717741 25839 webserver.cc:460] Webserver started at http://127.6.57.66:44341/ using document root <none> and password file <none>
I20260806 06:49:59.718001 25839 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:59.718070 25839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:59.718220 25839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:59.719276 25839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "c23226bbb82b4b9a904a53c13a3ce272"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "44ca17110e8eea6e7891f7c289420534"
server_key_iv: "a9d077e42dc600100621312d8003b7ad"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.719627 25839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "c23226bbb82b4b9a904a53c13a3ce272"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "44ca17110e8eea6e7891f7c289420534"
server_key_iv: "a9d077e42dc600100621312d8003b7ad"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.720868 25839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:49:59.721601 25855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.721786 25839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:49:59.721894 25839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "c23226bbb82b4b9a904a53c13a3ce272"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "44ca17110e8eea6e7891f7c289420534"
server_key_iv: "a9d077e42dc600100621312d8003b7ad"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.721966 25839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:59.735554 25839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.735826 25839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.736029 25839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.736325 25839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:59.736649 25839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:59.736706 25839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.736747 25839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:59.736775 25839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.742621 25839 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:43573
I20260806 06:49:59.742669 25973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:43573 every 8 connection(s)
I20260806 06:49:59.742969 25839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:49:59.747931 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25839
I20260806 06:49:59.748028 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:49:59.748051 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.743750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:35531 (local address 127.6.57.126:46565)
0806 06:49:59.743806 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.743808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.744003 (+ 195us) server_negotiation.cc:407] Connection header received
0806 06:49:59.744501 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.744502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.744514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.744565 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.745183 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.745678 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.746702 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.746883 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.746937 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.747778 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.747780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.747780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:59.747785 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.747804 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:59.747810 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.747873 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.747925 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.747952 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:49:59.748256 6373 external_mini_cluster.cc:1428] Setting key 6ee03d3b24a4c04452bbdde8a3682f1e
I20260806 06:49:59.748317 25976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.743723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46565 (local address 127.6.57.66:35531)
0806 06:49:59.743906 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.743938 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.744414 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.744618 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.744621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.744629 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:59.745107 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.745111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.745766 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.745767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.746629 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.746632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.746959 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.746960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.747088 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.747642 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:59.747643 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:59.747670 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:59.747691 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:59.747873 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:59.747875 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:59.747877 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.747924 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.748056 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260806 06:49:59.748836 25974 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46565
I20260806 06:49:59.748914 25974 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:59.749140 25974 heartbeater.cc:511] Master 127.6.57.126:46565 requested a full tablet report, sending...
I20260806 06:49:59.749485 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--builtin_ntp_servers=127.6.57.84:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 06:49:59.749563 25639 ts_manager.cc:194] Registered new tserver with Master: c23226bbb82b4b9a904a53c13a3ce272 (127.6.57.66:43573)
I20260806 06:49:59.750013 25639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.66:35531
W20260806 06:49:59.832854 25977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:49:59.833050 25977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:49:59.833070 25977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:49:59.834161 25977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:49:59.834200 25977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:49:59.834224 25977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:49:59.835783 25977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:49:59.836069 25977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:49:59.836403 25977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:49:59.841486 25985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.841547 25977 server_base.cc:1034] running on GCE node
W20260806 06:49:59.841475 25982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:49:59.841459 25983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:49:59.841806 25977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:49:59.842065 25977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:49:59.843250 25977 hybrid_clock.cc:648] HybridClock initialized: now 1785998999843234 us; error 27 us; skew 500 ppm
I20260806 06:49:59.851279 25977 webserver.cc:460] Webserver started at http://127.6.57.67:33971/ using document root <none> and password file <none>
I20260806 06:49:59.851532 25977 fs_manager.cc:359] Metadata directory not provided
I20260806 06:49:59.851603 25977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:49:59.851755 25977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:49:59.852854 25977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "71bc61b698d4450db1ff61b73bbcb91b"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "c1047e18604a281cc79999fd51e287ed"
server_key_iv: "ef0f78bdab116bc8da100f220d729ae5"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.853209 25977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "71bc61b698d4450db1ff61b73bbcb91b"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "c1047e18604a281cc79999fd51e287ed"
server_key_iv: "ef0f78bdab116bc8da100f220d729ae5"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.854468 25977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:49:59.855336 25991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.855623 25977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:49:59.855732 25977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "71bc61b698d4450db1ff61b73bbcb91b"
format_stamp: "Formatted at 2026-08-06 06:49:59 on dist-test-slave-5l3k"
server_key: "c1047e18604a281cc79999fd51e287ed"
server_key_iv: "ef0f78bdab116bc8da100f220d729ae5"
server_key_version: "encryptionkey@0"
I20260806 06:49:59.855805 25977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:49:59.875142 25977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:49:59.875458 25977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:49:59.875581 25977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:49:59.875777 25977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:49:59.876061 25977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:49:59.876096 25977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.876118 25977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:49:59.876133 25977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:49:59.882320 25977 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:45795
I20260806 06:49:59.882367 26110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:45795 every 8 connection(s)
I20260806 06:49:59.882761 25977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:49:59.884860 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 25977
I20260806 06:49:59.884991 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:49:59.885214 6373 external_mini_cluster.cc:1428] Setting key eb2e54324a600236edb3b3d77bc8adc7
I20260806 06:49:59.889060 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.883614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:58651 (local address 127.6.57.126:46565)
0806 06:49:59.883691 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.883693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.883950 (+ 257us) server_negotiation.cc:407] Connection header received
0806 06:49:59.884411 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.884412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.884425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.884584 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.885539 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.886297 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.887464 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.887698 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.887793 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.888729 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.888731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.888732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:59.888737 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.888753 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:59.888757 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.888837 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.888943 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.888973 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:49:59.889335 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.883587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:46565 (local address 127.6.57.67:58651)
0806 06:49:59.883784 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.883824 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.884302 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.884667 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.884671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.884680 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:49:59.885389 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.885396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.886401 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.886403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.887343 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.887345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.887805 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.887806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.887952 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.888572 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:49:59.888572 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:49:59.888601 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:49:59.888625 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:49:59.888840 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:49:59.888847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:49:59.888850 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.888946 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.889100 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260806 06:49:59.889837 26111 heartbeater.cc:347] Connected to a master server at 127.6.57.126:46565
I20260806 06:49:59.889950 26111 heartbeater.cc:464] Registering TS with master...
I20260806 06:49:59.890146 26111 heartbeater.cc:511] Master 127.6.57.126:46565 requested a full tablet report, sending...
I20260806 06:49:59.890590 25639 ts_manager.cc:194] Registered new tserver with Master: 71bc61b698d4450db1ff61b73bbcb91b (127.6.57.67:45795)
I20260806 06:49:59.890955 25639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.67:58651
I20260806 06:49:59.897209 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:49:59.901456 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.898690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.6.57.126:46565)
0806 06:49:59.898744 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.898746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.898890 (+ 144us) server_negotiation.cc:407] Connection header received
0806 06:49:59.898926 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.898926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.898934 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.898982 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:49:59.899168 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.899622 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.900343 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.900530 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.900577 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.901254 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:49:59.901255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:49:59.901255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:49:59.901259 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:49:59.901269 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:49:59.901272 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:49:59.901332 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.901396 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.901404 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:49:59.903235 25639 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:49:59.904734 25639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:49:59.911203 26122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.907090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:45795 (local address 127.0.0.1:58056)
0806 06:49:59.907402 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.907411 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.907469 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.907797 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.907798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.907800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.907831 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.907833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.908749 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.908750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.910014 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.910016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.910912 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.910988 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.911000 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.911061 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.911111 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":6,"thread_start_us":102,"threads_started":1}
I20260806 06:49:59.911401 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.907129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59464 (local address 127.6.57.66:43573)
0806 06:49:59.907336 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.907341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.907352 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:49:59.907394 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.907396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.907412 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.907922 (+ 510us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.907986 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.909097 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910228 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.910857 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910927 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.911021 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.911261 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.911308 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260806 06:49:59.911432 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.906919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43573 (local address 127.0.0.1:59464)
0806 06:49:59.907220 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.907230 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.907265 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.907539 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.907542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.907614 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.907661 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.907663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.909164 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.909165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.910117 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.910120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.911055 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.911056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.911117 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.911120 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.911170 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.911199 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":57,"threads_started":1}
I20260806 06:49:59.911419 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.907232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.6.57.67:45795)
0806 06:49:59.907382 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.907387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.907455 (+ 68us) server_negotiation.cc:407] Connection header received
0806 06:49:59.907543 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.907545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.907560 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.907806 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.907923 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.908665 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910103 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.910844 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910912 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.911169 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.911200 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.911235 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260806 06:49:59.911967 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.906882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:46563 (local address 127.0.0.1:43868)
0806 06:49:59.907007 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.907059 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.907125 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.907570 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.907571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.907573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.907659 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.907662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.908942 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.908943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.910014 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.910016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.911741 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.911742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.911800 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.911819 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.911864 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.911903 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260806 06:49:59.912266 26041 tablet_service.cc:1467] Processing CreateTablet for tablet bc5c418915d94691945ff69326ab7988 (DEFAULT_TABLE table=test-table [id=2cb8d1b8470a4bc0ae6e60758a1a4237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.912266 25906 tablet_service.cc:1467] Processing CreateTablet for tablet bc5c418915d94691945ff69326ab7988 (DEFAULT_TABLE table=test-table [id=2cb8d1b8470a4bc0ae6e60758a1a4237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.912643 25906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5c418915d94691945ff69326ab7988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.912650 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5c418915d94691945ff69326ab7988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.915302 26125 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Bootstrap starting.
I20260806 06:49:59.915721 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.906989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.6.57.65:46563)
0806 06:49:59.907388 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.907392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.907401 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:49:59.907440 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.907442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.907457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.907574 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.907742 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.908861 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.910569 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.911678 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.911741 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.915538 (+ 3797us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.915569 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.915624 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":86,"threads_started":1}
I20260806 06:49:59.915936 26126 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Bootstrap starting.
I20260806 06:49:59.916006 26125 tablet_bootstrap.cc:654] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.916328 26125 log.cc:824] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.916329 25767 tablet_service.cc:1467] Processing CreateTablet for tablet bc5c418915d94691945ff69326ab7988 (DEFAULT_TABLE table=test-table [id=2cb8d1b8470a4bc0ae6e60758a1a4237]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:49:59.916620 26126 tablet_bootstrap.cc:654] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.916630 25767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5c418915d94691945ff69326ab7988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:49:59.916896 26126 log.cc:824] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.917129 26125 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: No bootstrap required, opened a new log
I20260806 06:49:59.917186 26125 ts_tablet_manager.cc:1397] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:59.917706 26126 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: No bootstrap required, opened a new log
I20260806 06:49:59.917778 26126 ts_tablet_manager.cc:1397] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:59.919420 26129 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: Bootstrap starting.
I20260806 06:49:59.919997 26129 tablet_bootstrap.cc:654] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: Neither blocks nor log segments found. Creating new log.
I20260806 06:49:59.919934 26125 raft_consensus.cc:348] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.920110 26125 raft_consensus.cc:374] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.920148 26125 raft_consensus.cc:729] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71bc61b698d4450db1ff61b73bbcb91b, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.920277 26125 consensus_queue.cc:260] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.920240 26126 raft_consensus.cc:348] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.920316 26129 log.cc:824] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: Log is configured to *not* fsync() on all Append() calls
I20260806 06:49:59.920370 26126 raft_consensus.cc:374] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.920418 26126 raft_consensus.cc:729] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c23226bbb82b4b9a904a53c13a3ce272, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.920533 26125 ts_tablet_manager.cc:1428] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:49:59.920562 26126 consensus_queue.cc:260] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.920686 26111 heartbeater.cc:503] Master 127.6.57.126:46565 was elected leader, sending a full tablet report...
I20260806 06:49:59.920835 26126 ts_tablet_manager.cc:1428] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 06:49:59.920943 25974 heartbeater.cc:503] Master 127.6.57.126:46565 was elected leader, sending a full tablet report...
I20260806 06:49:59.921119 26129 tablet_bootstrap.cc:492] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: No bootstrap required, opened a new log
I20260806 06:49:59.921185 26129 ts_tablet_manager.cc:1397] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:49:59.923406 26129 raft_consensus.cc:348] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.923540 26129 raft_consensus.cc:374] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:49:59.923583 26129 raft_consensus.cc:729] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c08f8cb3b845fbae34ef5d04bcdfb9, State: Initialized, Role: FOLLOWER
I20260806 06:49:59.923683 26129 consensus_queue.cc:260] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.923916 26129 ts_tablet_manager.cc:1428] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:49:59.923988 25836 heartbeater.cc:503] Master 127.6.57.126:46565 was elected leader, sending a full tablet report...
I20260806 06:49:59.947988 26133 raft_consensus.cc:482] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:49:59.948130 26133 raft_consensus.cc:504] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.948441 26133 leader_election.cc:288] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71bc61b698d4450db1ff61b73bbcb91b (127.6.57.67:45795), c23226bbb82b4b9a904a53c13a3ce272 (127.6.57.66:43573)
I20260806 06:49:59.951166 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.948622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43573 (local address 127.6.57.65:55211)
0806 06:49:59.948679 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.948692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.948737 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.948940 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.948942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.948943 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.949014 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.949016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.949650 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.949651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.950366 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.950368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.950916 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.950917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.950987 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.950990 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.951058 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.951099 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:49:59.951165 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.948724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:55211 (local address 127.6.57.66:43573)
0806 06:49:59.948812 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.948814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.948821 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:49:59.948859 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.948860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.948867 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.948935 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.949104 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.949564 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.950466 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.950840 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.950916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.950926 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.951066 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.951099 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 06:49:59.951537 25926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5c418915d94691945ff69326ab7988" candidate_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23226bbb82b4b9a904a53c13a3ce272" is_pre_election: true
I20260806 06:49:59.951623 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.948648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:45795 (local address 127.6.57.65:56295)
0806 06:49:59.948860 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.948869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:49:59.948954 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:49:59.949132 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:49:59.949133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:49:59.949135 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:49:59.949207 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.949208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.949798 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.949799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.950706 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:49:59.950709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.951394 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.951396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:49:59.951481 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.951484 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:49:59.951534 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 06:49:59.951564 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":94,"threads_started":1}
I20260806 06:49:59.951665 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.948705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:56295 (local address 127.6.57.67:45795)
0806 06:49:59.948771 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.948773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.948940 (+ 167us) server_negotiation.cc:407] Connection header received
0806 06:49:59.949028 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.949029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.949044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.949117 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:49:59.949274 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.949713 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.950806 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.951182 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.951297 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.951309 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.951547 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.951598 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:49:59.951691 25926 raft_consensus.cc:2454] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c08f8cb3b845fbae34ef5d04bcdfb9 in term 0.
I20260806 06:49:59.951917 25719 leader_election.cc:302] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c08f8cb3b845fbae34ef5d04bcdfb9, c23226bbb82b4b9a904a53c13a3ce272; no voters:
I20260806 06:49:59.952008 26061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5c418915d94691945ff69326ab7988" candidate_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bc61b698d4450db1ff61b73bbcb91b" is_pre_election: true
I20260806 06:49:59.952059 26133 raft_consensus.cc:2781] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:49:59.952100 26133 raft_consensus.cc:482] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:49:59.952140 26133 raft_consensus.cc:3037] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.952152 26061 raft_consensus.cc:2454] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c08f8cb3b845fbae34ef5d04bcdfb9 in term 0.
I20260806 06:49:59.953025 26133 raft_consensus.cc:504] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.953168 26133 leader_election.cc:288] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [CANDIDATE]: Term 1 election: Requested vote from peers 71bc61b698d4450db1ff61b73bbcb91b (127.6.57.67:45795), c23226bbb82b4b9a904a53c13a3ce272 (127.6.57.66:43573)
I20260806 06:49:59.953311 25926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5c418915d94691945ff69326ab7988" candidate_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23226bbb82b4b9a904a53c13a3ce272"
I20260806 06:49:59.953315 26061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5c418915d94691945ff69326ab7988" candidate_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71bc61b698d4450db1ff61b73bbcb91b"
I20260806 06:49:59.953387 25926 raft_consensus.cc:3037] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.953430 26061 raft_consensus.cc:3037] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:49:59.954136 25926 raft_consensus.cc:2454] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c08f8cb3b845fbae34ef5d04bcdfb9 in term 1.
I20260806 06:49:59.954309 25719 leader_election.cc:302] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c08f8cb3b845fbae34ef5d04bcdfb9, c23226bbb82b4b9a904a53c13a3ce272; no voters:
I20260806 06:49:59.954406 26061 raft_consensus.cc:2454] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c08f8cb3b845fbae34ef5d04bcdfb9 in term 1.
I20260806 06:49:59.954456 26133 raft_consensus.cc:2781] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:49:59.954644 26133 raft_consensus.cc:686] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 1 LEADER]: Becoming Leader. State: Replica: 66c08f8cb3b845fbae34ef5d04bcdfb9, State: Running, Role: LEADER
I20260806 06:49:59.954744 26133 consensus_queue.cc:237] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } }
I20260806 06:49:59.955530 25640 catalog_manager.cc:5495] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66c08f8cb3b845fbae34ef5d04bcdfb9 (127.6.57.65). New cstate: current_term: 1 leader_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c08f8cb3b845fbae34ef5d04bcdfb9" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 46563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 } health_report { overall_health: UNKNOWN } } }
I20260806 06:49:59.969244 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.966601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.6.57.65:46563)
0806 06:49:59.966651 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:49:59.966653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:49:59.966664 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:49:59.966706 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:49:59.966707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:49:59.966727 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:49:59.966827 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:49:59.967009 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.967534 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.968351 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:49:59.968580 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:49:59.968663 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:49:59.968845 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:49:59.968947 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:49:59.969072 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 06:49:59.969124 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:49:59.969136 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 06:49:59.971045 25926 raft_consensus.cc:1261] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 1 FOLLOWER]: Refusing update from remote peer 66c08f8cb3b845fbae34ef5d04bcdfb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:59.971045 26061 raft_consensus.cc:1261] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 1 FOLLOWER]: Refusing update from remote peer 66c08f8cb3b845fbae34ef5d04bcdfb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:49:59.971419 26133 consensus_queue.cc:1035] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c23226bbb82b4b9a904a53c13a3ce272" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:59.971570 26135 consensus_queue.cc:1035] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71bc61b698d4450db1ff61b73bbcb91b" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 45795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:49:59.975106 26140 mvcc.cc:205] Tried to move back new op lower bound from 7315451903878713344 to 7315451903815286784. Current Snapshot: MvccSnapshot[applied={T|T < 7315451903878713344 or (T in {7315451903878713344})}]
I20260806 06:49:59.976522 25640 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:59.976617 25640 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:49:59.977463 25640 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d7187704fac4221921cd05e786e9ccf: Sending DeleteTablet for 3 replicas of tablet bc5c418915d94691945ff69326ab7988
I20260806 06:49:59.977852 25767 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5c418915d94691945ff69326ab7988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='slave'} at 127.0.0.1:43868
I20260806 06:49:59.977950 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5c418915d94691945ff69326ab7988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='slave'} at 127.0.0.1:58056
I20260806 06:49:59.978078 25906 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5c418915d94691945ff69326ab7988 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:49:59 UTC) from {username='slave'} at 127.0.0.1:59464
I20260806 06:49:59.978142 26147 tablet_replica.cc:330] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9: stopping tablet replica
I20260806 06:49:59.978250 26147 raft_consensus.cc:2229] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:49:59.978354 26147 raft_consensus.cc:2258] T bc5c418915d94691945ff69326ab7988 P 66c08f8cb3b845fbae34ef5d04bcdfb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:59.978397 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25702
I20260806 06:49:59.982667 26149 tablet_replica.cc:330] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: stopping tablet replica
I20260806 06:49:59.982786 26149 raft_consensus.cc:2229] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:59.982920 26149 raft_consensus.cc:2258] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:59.983078 26148 tablet_replica.cc:330] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: stopping tablet replica
I20260806 06:49:59.983182 26148 raft_consensus.cc:2229] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:49:59.983294 26148 raft_consensus.cc:2258] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:49:59.983435 26149 ts_tablet_manager.cc:1905] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:59.983748 26148 ts_tablet_manager.cc:1905] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:49:59.985250 26149 ts_tablet_manager.cc:1918] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:59.985323 26149 log.cc:1196] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/bc5c418915d94691945ff69326ab7988
I20260806 06:49:59.985514 26148 ts_tablet_manager.cc:1918] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:49:59.985666 26148 log.cc:1196] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/bc5c418915d94691945ff69326ab7988
W20260806 06:49:59.985672 25624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:46563 (error 108)
I20260806 06:49:59.985692 26149 ts_tablet_manager.cc:1939] T bc5c418915d94691945ff69326ab7988 P c23226bbb82b4b9a904a53c13a3ce272: Deleting consensus metadata
I20260806 06:49:59.985703 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25839
I20260806 06:49:59.986794 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.986640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:46563 (local address 127.0.0.1:43888)
0806 06:49:59.986706 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.986752 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:46563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 06:49:59.986965 25624 catalog_manager.cc:4584] TS 66c08f8cb3b845fbae34ef5d04bcdfb9 (127.6.57.65:46563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc5c418915d94691945ff69326ab7988: Network error: Client connection negotiation failed: client connection to 127.6.57.65:46563: connect: Connection refused (error 111)
I20260806 06:49:59.987739 26148 ts_tablet_manager.cc:1939] T bc5c418915d94691945ff69326ab7988 P 71bc61b698d4450db1ff61b73bbcb91b: Deleting consensus metadata
I20260806 06:49:59.988171 25624 catalog_manager.cc:4841] TS 71bc61b698d4450db1ff61b73bbcb91b (127.6.57.67:45795): tablet bc5c418915d94691945ff69326ab7988 (table test-table [id=2cb8d1b8470a4bc0ae6e60758a1a4237]) successfully deleted
I20260806 06:49:59.991355 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25977
I20260806 06:49:59.991369 26119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:49:59.991243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:43573 (local address 127.0.0.1:59474)
0806 06:49:59.991307 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 06:49:59.991338 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.66:43573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 06:49:59.996562 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 25609
2026-08-06T06:50:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (711 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:00 dist-test-slave-5l3k krb5kdc[26153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:00 dist-test-slave-5l3k krb5kdc[26153](info): set up 2 sockets
Aug 06 06:50:00 dist-test-slave-5l3k krb5kdc[26153](info): commencing operation
krb5kdc: starting...
W20260806 06:50:01.055444 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:01.173364 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34319
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:34319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:01.252663 26168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:01.252873 26168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:01.252908 26168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:01.253998 26168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:01.254061 26168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:01.254084 26168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:01.254124 26168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:01.254153 26168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:01.255723 26168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:34319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:34319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:01.255988 26168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:01.256220 26168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:01.261296 26174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.261276 26173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.261315 26177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:01.261809 26168 server_base.cc:1034] running on GCE node
I20260806 06:50:01.262096 26168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:01.262440 26168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:01.263624 26168 hybrid_clock.cc:648] HybridClock initialized: now 1785999001263611 us; error 27 us; skew 500 ppm
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:01.265919 26168 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:01.266371 26168 webserver.cc:460] Webserver started at http://127.6.57.126:36241/ using document root <none> and password file <none>
I20260806 06:50:01.266628 26168 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:01.266674 26168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:01.266762 26168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:01.267707 26168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "50b15bfd10d345fab7c58089ac670219"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "d942923c990db3b8726ab81d81d37817"
server_key_iv: "1ffcc13cce0e5dfd88927a255c851f05"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.268010 26168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "50b15bfd10d345fab7c58089ac670219"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "d942923c990db3b8726ab81d81d37817"
server_key_iv: "1ffcc13cce0e5dfd88927a255c851f05"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.269230 26168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:01.269896 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.270044 26168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.270130 26168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "50b15bfd10d345fab7c58089ac670219"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "d942923c990db3b8726ab81d81d37817"
server_key_iv: "1ffcc13cce0e5dfd88927a255c851f05"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.270184 26168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:01.296088 26168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:01.298557 26168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:01.298681 26168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:01.302639 26168 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:34319
I20260806 06:50:01.302743 26240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:34319 every 8 connection(s)
I20260806 06:50:01.303062 26168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:01.303997 26241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:01.306402 26241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: Bootstrap starting.
I20260806 06:50:01.307070 26241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:01.307387 26241 log.cc:824] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:01.307668 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26168
I20260806 06:50:01.307775 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:01.307978 6373 external_mini_cluster.cc:1428] Setting key f368b816b327999258409237abf9523d
I20260806 06:50:01.308246 26241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: No bootstrap required, opened a new log
I20260806 06:50:01.310614 26241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } }
I20260806 06:50:01.310822 26241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:01.310868 26241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50b15bfd10d345fab7c58089ac670219, State: Initialized, Role: FOLLOWER
I20260806 06:50:01.311018 26241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } }
I20260806 06:50:01.311101 26241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:01.311143 26241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:01.311192 26241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:01.311967 26241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } }
I20260806 06:50:01.312084 26241 leader_election.cc:302] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50b15bfd10d345fab7c58089ac670219; no voters:
I20260806 06:50:01.312239 26241 leader_election.cc:288] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:01.312300 26247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:01.312465 26247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [term 1 LEADER]: Becoming Leader. State: Replica: 50b15bfd10d345fab7c58089ac670219, State: Running, Role: LEADER
I20260806 06:50:01.312508 26241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:01.312551 26247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } }
I20260806 06:50:01.312884 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50b15bfd10d345fab7c58089ac670219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } } }
I20260806 06:50:01.312876 26247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50b15bfd10d345fab7c58089ac670219. Latest consensus state: current_term: 1 leader_uuid: "50b15bfd10d345fab7c58089ac670219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50b15bfd10d345fab7c58089ac670219" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 34319 } } }
I20260806 06:50:01.312968 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:01.312974 26247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:01.313176 26253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:01.313702 26253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:01.316455 26253 catalog_manager.cc:1347] Generated new cluster ID: 501aa57821d24bf8a528001e14c67732
I20260806 06:50:01.316524 26253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:01.320106 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.308481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.6.57.126:34319)
0806 06:50:01.308760 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.308770 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.308808 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:50:01.309704 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.309715 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.309780 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.309941 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:01.310548 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.312329 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.313554 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.314182 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.314303 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.316902 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:01.316906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:01.316906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:01.316917 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:01.318574 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.318941 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.318944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.318945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.319053 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.319272 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.319273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.319273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.319501 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:01.319677 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.319745 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.319881 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":129,"threads_started":1}
I20260806 06:50:01.323486 26253 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:01.324115 26253 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:01.328008 26253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: Generated new TSK 0
I20260806 06:50:01.328189 26253 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.57.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
Entry for principal kudu/127.6.57.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab.
I20260806 06:50:01.363394 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:01.455149 26268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:01.455354 26268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:01.455391 26268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:01.456498 26268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:01.456568 26268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:01.456615 26268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:50:01.458285 26268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.65.keytab
--principal=kudu/127.6.57.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:01.458563 26268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:01.458798 26268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:01.464484 26274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.464497 26277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.464498 26275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:01.464952 26268 server_base.cc:1034] running on GCE node
I20260806 06:50:01.465173 26268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:01.465425 26268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:01.466571 26268 hybrid_clock.cc:648] HybridClock initialized: now 1785999001466562 us; error 38 us; skew 500 ppm
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:01.469238 26268 init.cc:377] Logged in from keytab as kudu/127.6.57.65@KRBTEST.COM (short username kudu)
I20260806 06:50:01.469731 26268 webserver.cc:460] Webserver started at http://127.6.57.65:37189/ using document root <none> and password file <none>
I20260806 06:50:01.469998 26268 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:01.470082 26268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:01.470233 26268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:01.471354 26268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "637d9e21cbb74e8e879c63316c3b37d1"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "0f5012d26889b8e2d3b46a5f645b55b7"
server_key_iv: "e47464722e215548bd08aaaff05f5007"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.471707 26268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "637d9e21cbb74e8e879c63316c3b37d1"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "0f5012d26889b8e2d3b46a5f645b55b7"
server_key_iv: "e47464722e215548bd08aaaff05f5007"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.473398 26268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:01.474237 26284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.474432 26268 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:01.474578 26268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "637d9e21cbb74e8e879c63316c3b37d1"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "0f5012d26889b8e2d3b46a5f645b55b7"
server_key_iv: "e47464722e215548bd08aaaff05f5007"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.474658 26268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:01.502324 26268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:01.504347 26268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:01.504539 26268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:01.504804 26268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:01.505146 26268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:01.505208 26268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.505254 26268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:01.505281 26268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.511471 26268 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:34641
I20260806 06:50:01.511521 26403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:34641 every 8 connection(s)
I20260806 06:50:01.511844 26268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.65@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:01.519035 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26268
I20260806 06:50:01.519168 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:01.519374 6373 external_mini_cluster.cc:1428] Setting key 257a38f842a392c8f99e40754e717f9d
I20260806 06:50:01.521781 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.512628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:59145 (local address 127.6.57.126:34319)
0806 06:50:01.512705 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.512708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.512933 (+ 225us) server_negotiation.cc:407] Connection header received
0806 06:50:01.513613 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.513614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.513627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.513697 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:01.514766 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.515254 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.516218 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.516437 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.516486 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.518685 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:01.518686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:01.518686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:01.518692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:01.520444 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.520836 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.520837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.520837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.520899 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.521153 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.521154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.521154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.521261 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:01.521336 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.521597 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.521693 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 06:50:01.521934 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.512616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34319 (local address 127.6.57.65:59145)
0806 06:50:01.512801 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.512840 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.513453 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.513757 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.513761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.514372 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:01.514685 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.514691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.515335 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.515336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.516149 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.516150 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.516506 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.516507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.516617 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.517057 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:01.517065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:01.518601 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:01.520529 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.520536 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.520539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.520758 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.520966 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.520967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.520968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.521077 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.521328 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:01.521335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:01.521468 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.521565 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.521715 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260806 06:50:01.522395 26404 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34319
I20260806 06:50:01.522540 26404 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:01.522738 26404 heartbeater.cc:511] Master 127.6.57.126:34319 requested a full tablet report, sending...
I20260806 06:50:01.523298 26203 ts_manager.cc:194] Registered new tserver with Master: 637d9e21cbb74e8e879c63316c3b37d1 (127.6.57.65:34641)
I20260806 06:50:01.524255 26203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.65@KRBTEST.COM'} at 127.6.57.65:59145
WARNING: no policy specified for kudu/127.6.57.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
Entry for principal kudu/127.6.57.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab.
I20260806 06:50:01.545763 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:01.626741 26410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:01.626926 26410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:01.626948 26410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:01.628122 26410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:01.628162 26410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:01.628187 26410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:50:01.629585 26410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.66.keytab
--principal=kudu/127.6.57.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:01.629766 26410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:01.630024 26410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:01.634747 26418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.634753 26415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.634753 26416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:01.635110 26410 server_base.cc:1034] running on GCE node
I20260806 06:50:01.635319 26410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:01.635552 26410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:01.636695 26410 hybrid_clock.cc:648] HybridClock initialized: now 1785999001636661 us; error 46 us; skew 500 ppm
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:01.639194 26410 init.cc:377] Logged in from keytab as kudu/127.6.57.66@KRBTEST.COM (short username kudu)
I20260806 06:50:01.639634 26410 webserver.cc:460] Webserver started at http://127.6.57.66:46773/ using document root <none> and password file <none>
I20260806 06:50:01.639889 26410 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:01.639971 26410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:01.640125 26410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:01.641059 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "bfc3299ed58746b580596151e41ce15a"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "fc51c0ece8a486fa47297a077db40b24"
server_key_iv: "c419ddb2ac8a8963a6e62a4f43fccc02"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.641395 26410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "bfc3299ed58746b580596151e41ce15a"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "fc51c0ece8a486fa47297a077db40b24"
server_key_iv: "c419ddb2ac8a8963a6e62a4f43fccc02"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.642647 26410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:01.643353 26426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.643519 26410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:01.643615 26410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "bfc3299ed58746b580596151e41ce15a"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "fc51c0ece8a486fa47297a077db40b24"
server_key_iv: "c419ddb2ac8a8963a6e62a4f43fccc02"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.643693 26410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:01.667871 26410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:01.669636 26410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:01.669795 26410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:01.670033 26410 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:01.670333 26410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:01.670387 26410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.670426 26410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:01.670452 26410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.676518 26410 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:33377
I20260806 06:50:01.676570 26545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:33377 every 8 connection(s)
I20260806 06:50:01.676860 26410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:01.680670 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26410
I20260806 06:50:01.680773 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:01.680955 6373 external_mini_cluster.cc:1428] Setting key d67beac6c28eacd06d03502d579e210e
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.66@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:01.686928 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.677584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:40669 (local address 127.6.57.126:34319)
0806 06:50:01.677642 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.677659 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.677851 (+ 192us) server_negotiation.cc:407] Connection header received
0806 06:50:01.678422 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.678423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.678436 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.678540 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:01.679453 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.679940 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.681115 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.681388 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.681453 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.683967 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:01.683968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:01.683969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:01.683976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:01.685368 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.685890 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.685891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.685892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.685952 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.686245 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.686246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.686246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.686368 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:01.686429 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.686799 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.686863 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:50:01.687222 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.677519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34319 (local address 127.6.57.66:40669)
0806 06:50:01.677720 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.677771 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.678284 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.678625 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.678629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.679186 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:01.679376 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.679380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.680015 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.680016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.681034 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.681038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.681465 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.681466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.681643 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.682127 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:01.682137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:01.683857 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:01.685449 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.685458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.685461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.685801 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.686032 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.686033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.686033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.686176 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.686443 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:01.686450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:01.686643 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.686765 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.686947 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260806 06:50:01.687839 26546 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34319
I20260806 06:50:01.687945 26546 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:01.688138 26546 heartbeater.cc:511] Master 127.6.57.126:34319 requested a full tablet report, sending...
I20260806 06:50:01.688571 26203 ts_manager.cc:194] Registered new tserver with Master: bfc3299ed58746b580596151e41ce15a (127.6.57.66:33377)
I20260806 06:50:01.689096 26203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.66@KRBTEST.COM'} at 127.6.57.66:40669
WARNING: no policy specified for kudu/127.6.57.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
Entry for principal kudu/127.6.57.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab.
I20260806 06:50:01.707334 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:01.790619 26551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:01.790807 26551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:01.790829 26551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:01.791922 26551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:01.791961 26551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:01.791986 26551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:50:01.793336 26551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.67.keytab
--principal=kudu/127.6.57.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:34319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:01.793507 26551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:01.793681 26551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:01.798612 26557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.798586 26558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:01.798601 26560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:01.798877 26551 server_base.cc:1034] running on GCE node
I20260806 06:50:01.799110 26551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:01.799367 26551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:01.800539 26551 hybrid_clock.cc:648] HybridClock initialized: now 1785999001800540 us; error 44 us; skew 500 ppm
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:01.803072 26551 init.cc:377] Logged in from keytab as kudu/127.6.57.67@KRBTEST.COM (short username kudu)
I20260806 06:50:01.803515 26551 webserver.cc:460] Webserver started at http://127.6.57.67:36575/ using document root <none> and password file <none>
I20260806 06:50:01.803694 26551 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:01.803732 26551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:01.803819 26551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:01.804768 26551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "48be65b9feb54a798a6a08207b4d1177"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "63bdace8040fd80d090dfc48f42eb68a"
server_key_iv: "589cfc55da0f7054a001911e2e7c109c"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.805059 26551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "48be65b9feb54a798a6a08207b4d1177"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "63bdace8040fd80d090dfc48f42eb68a"
server_key_iv: "589cfc55da0f7054a001911e2e7c109c"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.806270 26551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:01.807003 26567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.807163 26551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.807247 26551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "48be65b9feb54a798a6a08207b4d1177"
format_stamp: "Formatted at 2026-08-06 06:50:01 on dist-test-slave-5l3k"
server_key: "63bdace8040fd80d090dfc48f42eb68a"
server_key_iv: "589cfc55da0f7054a001911e2e7c109c"
server_key_version: "encryptionkey@0"
I20260806 06:50:01.807300 26551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:01.825232 26551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:01.827157 26551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:01.827268 26551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:01.827451 26551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:01.827759 26551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:01.827828 26551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.827868 26551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:01.827890 26551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:01.833506 26551 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:35373
I20260806 06:50:01.833581 26687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:35373 every 8 connection(s)
I20260806 06:50:01.833909 26551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
Aug 06 06:50:01 dist-test-slave-5l3k krb5kdc[26153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999001, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.67@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:01.842109 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26551
I20260806 06:50:01.842221 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:01.842444 6373 external_mini_cluster.cc:1428] Setting key 499786c22e25f2272327d662de049ca0
I20260806 06:50:01.842861 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.834684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:60365 (local address 127.6.57.126:34319)
0806 06:50:01.834758 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.834760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.834950 (+ 190us) server_negotiation.cc:407] Connection header received
0806 06:50:01.835405 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.835406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.835416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.835467 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:01.836328 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.836815 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.837796 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.837981 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.838033 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.840169 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:01.840171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:01.840171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:01.840177 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:01.841339 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.841738 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.841739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.841740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.841806 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.842037 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.842038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.842038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.842202 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:01.842293 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.842665 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.842766 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 06:50:01.843053 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.834659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:34319 (local address 127.6.57.67:60365)
0806 06:50:01.834837 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.834871 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.835327 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.835517 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.835520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.836067 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:01.836253 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.836258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.836892 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.836892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.837704 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.837706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.838070 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.838071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.838167 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.838720 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:01.838736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:01.840088 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:01.841414 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.841420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.841422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.841645 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.841877 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:01.841878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:01.841878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:01.841967 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:01.842320 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:01.842326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:01.842449 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.842625 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.842816 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260806 06:50:01.843501 26688 heartbeater.cc:347] Connected to a master server at 127.6.57.126:34319
I20260806 06:50:01.843614 26688 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:01.843775 26688 heartbeater.cc:511] Master 127.6.57.126:34319 requested a full tablet report, sending...
I20260806 06:50:01.844205 26203 ts_manager.cc:194] Registered new tserver with Master: 48be65b9feb54a798a6a08207b4d1177 (127.6.57.67:35373)
I20260806 06:50:01.844568 26203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.57.67@KRBTEST.COM'} at 127.6.57.67:60365
I20260806 06:50:01.854291 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:50:01.861724 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.855719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.6.57.126:34319)
0806 06:50:01.855784 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.855786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.855963 (+ 177us) server_negotiation.cc:407] Connection header received
0806 06:50:01.855987 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.855990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.856002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.856048 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:01.856585 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.857065 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.857777 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.857959 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.858030 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.859175 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:01.859176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:01.859177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:01.859183 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:01.860754 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.861062 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.861062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.861063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.861111 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:01.861267 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:01.861267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:01.861268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:01.861403 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:01.861504 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.861575 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.861633 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:50:01.863196 26203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:01.864782 26203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:01.871220 26701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33377 (local address 127.0.0.1:53454)
0806 06:50:01.867202 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.867239 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.867310 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.867633 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.867636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.867637 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:01.867703 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.867704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.868793 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.868794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.869990 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.869993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.870856 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.870857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.870937 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.870951 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.871064 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.871128 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260806 06:50:01.871333 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.6.57.65:34641)
0806 06:50:01.867861 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.867865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.867874 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:01.867909 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.867911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.867924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.867991 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:01.868184 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.869203 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.870270 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.870977 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.871062 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.871125 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.871209 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.871255 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":98,"threads_started":1}
I20260806 06:50:01.871361 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:35373 (local address 127.0.0.1:58676)
0806 06:50:01.867283 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.867292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.867333 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.867920 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.867921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.867923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:01.867970 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.867971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.869179 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.869180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.870291 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.870294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.871137 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.871139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.871216 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.871221 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.871252 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.871288 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":24,"thread_start_us":83,"threads_started":1}
I20260806 06:50:01.871417 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.6.57.66:33377)
0806 06:50:01.867431 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.867435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.867447 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:50:01.867485 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.867487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.867502 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.867588 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:01.867789 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.868689 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.870106 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.870769 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.870855 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.870914 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.871276 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.871328 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
I20260806 06:50:01.871824 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.6.57.67:35373)
0806 06:50:01.867760 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.867764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.867773 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:01.867804 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.867806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.867825 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.867906 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:01.868030 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.869069 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.870366 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.871042 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.871584 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.871649 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.871675 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.871715 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260806 06:50:01.872251 26475 tablet_service.cc:1467] Processing CreateTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 (DEFAULT_TABLE table=test-table [id=800eaf2f96cd4691a22716efbfb6c321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:01.872642 26475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff9b911a80c4393b8caf99a6beec9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:01.872546 26617 tablet_service.cc:1467] Processing CreateTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 (DEFAULT_TABLE table=test-table [id=800eaf2f96cd4691a22716efbfb6c321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:01.872917 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff9b911a80c4393b8caf99a6beec9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:01.875463 26707 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Bootstrap starting.
I20260806 06:50:01.875785 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.867343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34641 (local address 127.0.0.1:39744)
0806 06:50:01.867449 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.867458 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.867497 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.868067 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.868068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.868070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:01.868105 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.868107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.869329 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.869330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.870177 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.870180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.871074 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.871075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.871129 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.871133 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.871192 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.871224 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":74,"threads_started":1}
I20260806 06:50:01.875813 26708 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Bootstrap starting.
I20260806 06:50:01.876186 26707 tablet_bootstrap.cc:654] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:01.876453 26708 tablet_bootstrap.cc:654] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:01.876463 26333 tablet_service.cc:1467] Processing CreateTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 (DEFAULT_TABLE table=test-table [id=800eaf2f96cd4691a22716efbfb6c321]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:01.876807 26333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fff9b911a80c4393b8caf99a6beec9b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:01.876811 26707 log.cc:824] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:01.877061 26708 log.cc:824] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:01.877669 26707 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: No bootstrap required, opened a new log
I20260806 06:50:01.877759 26707 ts_tablet_manager.cc:1397] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:01.877852 26708 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: No bootstrap required, opened a new log
I20260806 06:50:01.877964 26708 ts_tablet_manager.cc:1397] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:01.879599 26708 raft_consensus.cc:348] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.879767 26708 raft_consensus.cc:374] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:01.879812 26708 raft_consensus.cc:729] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48be65b9feb54a798a6a08207b4d1177, State: Initialized, Role: FOLLOWER
I20260806 06:50:01.879787 26707 raft_consensus.cc:348] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.879966 26707 raft_consensus.cc:374] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:01.879941 26708 consensus_queue.cc:260] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.880015 26707 raft_consensus.cc:729] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc3299ed58746b580596151e41ce15a, State: Initialized, Role: FOLLOWER
I20260806 06:50:01.880133 26707 consensus_queue.cc:260] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.880237 26708 ts_tablet_manager.cc:1428] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 06:50:01.880367 26688 heartbeater.cc:503] Master 127.6.57.126:34319 was elected leader, sending a full tablet report...
I20260806 06:50:01.880383 26707 ts_tablet_manager.cc:1428] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:50:01.880517 26546 heartbeater.cc:503] Master 127.6.57.126:34319 was elected leader, sending a full tablet report...
I20260806 06:50:01.880504 26711 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: Bootstrap starting.
I20260806 06:50:01.881162 26711 tablet_bootstrap.cc:654] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:01.881708 26711 log.cc:824] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:01.882302 26711 tablet_bootstrap.cc:492] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: No bootstrap required, opened a new log
I20260806 06:50:01.882350 26711 ts_tablet_manager.cc:1397] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:01.883900 26711 raft_consensus.cc:348] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.884039 26711 raft_consensus.cc:374] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:01.884060 26711 raft_consensus.cc:729] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637d9e21cbb74e8e879c63316c3b37d1, State: Initialized, Role: FOLLOWER
I20260806 06:50:01.884187 26711 consensus_queue.cc:260] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.884439 26711 ts_tablet_manager.cc:1428] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 06:50:01.884493 26404 heartbeater.cc:503] Master 127.6.57.126:34319 was elected leader, sending a full tablet report...
I20260806 06:50:01.901166 26712 raft_consensus.cc:482] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:01.901268 26712 raft_consensus.cc:504] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.901590 26712 leader_election.cc:288] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfc3299ed58746b580596151e41ce15a (127.6.57.66:33377), 637d9e21cbb74e8e879c63316c3b37d1 (127.6.57.65:34641)
I20260806 06:50:01.904397 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.901721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33377 (local address 127.6.57.67:55913)
0806 06:50:01.901801 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.901812 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.901852 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.901995 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.901996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.901998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:01.902065 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.902066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.902868 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.902870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.903578 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.903580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.904146 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.904147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.904233 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.904236 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.904297 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.904334 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 06:50:01.904394 26703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.901817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:55913 (local address 127.6.57.66:33377)
0806 06:50:01.901861 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.901863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.901888 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:50:01.901909 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.901910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.901919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.901981 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:01.902152 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.902806 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.903669 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.904045 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.904125 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.904138 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.904302 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.904336 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:01.904500 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.901803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:48499 (local address 127.6.57.65:34641)
0806 06:50:01.901907 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.901910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.901944 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:50:01.902003 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.902004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.902013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.902046 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:01.902205 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.902835 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.903760 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.904133 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.904237 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.904247 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.904401 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.904435 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 06:50:01.904572 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.901760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34641 (local address 127.6.57.67:48499)
0806 06:50:01.901885 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.901896 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:01.901951 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:01.902070 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:01.902071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:01.902073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:01.902138 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.902139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.902907 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.902908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.903666 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:01.903668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.904242 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.904243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:01.904315 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.904325 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 06:50:01.904395 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:50:01.904431 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260806 06:50:01.904697 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff9b911a80c4393b8caf99a6beec9b0" candidate_uuid: "48be65b9feb54a798a6a08207b4d1177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc3299ed58746b580596151e41ce15a" is_pre_election: true
I20260806 06:50:01.904839 26496 raft_consensus.cc:2454] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48be65b9feb54a798a6a08207b4d1177 in term 0.
I20260806 06:50:01.905012 26355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff9b911a80c4393b8caf99a6beec9b0" candidate_uuid: "48be65b9feb54a798a6a08207b4d1177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "637d9e21cbb74e8e879c63316c3b37d1" is_pre_election: true
I20260806 06:50:01.905030 26570 leader_election.cc:302] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48be65b9feb54a798a6a08207b4d1177, bfc3299ed58746b580596151e41ce15a; no voters:
I20260806 06:50:01.905138 26355 raft_consensus.cc:2454] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48be65b9feb54a798a6a08207b4d1177 in term 0.
I20260806 06:50:01.905171 26712 raft_consensus.cc:2781] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:01.905220 26712 raft_consensus.cc:482] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:01.905238 26712 raft_consensus.cc:3037] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:01.905889 26712 raft_consensus.cc:504] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.906016 26712 leader_election.cc:288] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [CANDIDATE]: Term 1 election: Requested vote from peers bfc3299ed58746b580596151e41ce15a (127.6.57.66:33377), 637d9e21cbb74e8e879c63316c3b37d1 (127.6.57.65:34641)
I20260806 06:50:01.906152 26355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff9b911a80c4393b8caf99a6beec9b0" candidate_uuid: "48be65b9feb54a798a6a08207b4d1177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "637d9e21cbb74e8e879c63316c3b37d1"
I20260806 06:50:01.906173 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fff9b911a80c4393b8caf99a6beec9b0" candidate_uuid: "48be65b9feb54a798a6a08207b4d1177" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc3299ed58746b580596151e41ce15a"
I20260806 06:50:01.906226 26355 raft_consensus.cc:3037] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:01.906231 26496 raft_consensus.cc:3037] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:01.907080 26355 raft_consensus.cc:2454] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48be65b9feb54a798a6a08207b4d1177 in term 1.
I20260806 06:50:01.907080 26496 raft_consensus.cc:2454] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48be65b9feb54a798a6a08207b4d1177 in term 1.
I20260806 06:50:01.907246 26570 leader_election.cc:302] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48be65b9feb54a798a6a08207b4d1177, bfc3299ed58746b580596151e41ce15a; no voters:
I20260806 06:50:01.907333 26712 raft_consensus.cc:2781] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:01.907467 26712 raft_consensus.cc:686] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 1 LEADER]: Becoming Leader. State: Replica: 48be65b9feb54a798a6a08207b4d1177, State: Running, Role: LEADER
I20260806 06:50:01.907531 26712 consensus_queue.cc:237] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } }
I20260806 06:50:01.908234 26203 catalog_manager.cc:5495] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48be65b9feb54a798a6a08207b4d1177 (127.6.57.67). New cstate: current_term: 1 leader_uuid: "48be65b9feb54a798a6a08207b4d1177" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48be65b9feb54a798a6a08207b4d1177" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 35373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 } health_report { overall_health: UNKNOWN } } }
I20260806 06:50:01.916213 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.914131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.6.57.67:35373)
0806 06:50:01.914196 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:01.914198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:01.914206 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:01.914239 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:01.914240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:01.914256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:01.914313 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:01.914426 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.914895 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.915587 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:01.915752 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:01.915802 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:01.915952 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:01.916037 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:01.916100 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:01.916140 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 06:50:01.916155 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 06:50:01.917758 26355 raft_consensus.cc:1261] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 1 FOLLOWER]: Refusing update from remote peer 48be65b9feb54a798a6a08207b4d1177: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:01.917758 26496 raft_consensus.cc:1261] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 1 FOLLOWER]: Refusing update from remote peer 48be65b9feb54a798a6a08207b4d1177: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:01.918130 26719 consensus_queue.cc:1035] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc3299ed58746b580596151e41ce15a" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 33377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:01.918253 26712 consensus_queue.cc:1035] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [LEADER]: Connected to new peer: Peer: permanent_uuid: "637d9e21cbb74e8e879c63316c3b37d1" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:01.920504 26724 mvcc.cc:205] Tried to move back new op lower bound from 7315451911852818432 to 7315451911813980160. Current Snapshot: MvccSnapshot[applied={T|T < 7315451911852818432}]
I20260806 06:50:01.920624 26723 mvcc.cc:205] Tried to move back new op lower bound from 7315451911852818432 to 7315451911813980160. Current Snapshot: MvccSnapshot[applied={T|T < 7315451911852818432}]
I20260806 06:50:01.922729 26203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:01.922852 26203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:01.923545 26203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50b15bfd10d345fab7c58089ac670219: Sending DeleteTablet for 3 replicas of tablet fff9b911a80c4393b8caf99a6beec9b0
I20260806 06:50:01.923852 26475 tablet_service.cc:1514] Processing DeleteTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:01 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:53454
I20260806 06:50:01.923862 26617 tablet_service.cc:1514] Processing DeleteTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:01 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:58676
I20260806 06:50:01.923907 26333 tablet_service.cc:1514] Processing DeleteTablet for tablet fff9b911a80c4393b8caf99a6beec9b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:01 UTC) from {username='kudu', principal='kudu/127.6.57.126@KRBTEST.COM'} at 127.0.0.1:39744
I20260806 06:50:01.924085 26733 tablet_replica.cc:330] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: stopping tablet replica
I20260806 06:50:01.924197 26733 raft_consensus.cc:2229] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:01.924269 26734 tablet_replica.cc:330] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1: stopping tablet replica
I20260806 06:50:01.924306 26733 raft_consensus.cc:2258] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:01.924425 26734 raft_consensus.cc:2229] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:01.924455 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26268
I20260806 06:50:01.924638 26734 raft_consensus.cc:2258] T fff9b911a80c4393b8caf99a6beec9b0 P 637d9e21cbb74e8e879c63316c3b37d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:01.925367 26733 ts_tablet_manager.cc:1905] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:01.926992 26733 ts_tablet_manager.cc:1918] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:01.927062 26733 log.cc:1196] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/fff9b911a80c4393b8caf99a6beec9b0
I20260806 06:50:01.927323 26733 ts_tablet_manager.cc:1939] T fff9b911a80c4393b8caf99a6beec9b0 P 48be65b9feb54a798a6a08207b4d1177: Deleting consensus metadata
I20260806 06:50:01.928112 26185 catalog_manager.cc:4841] TS 48be65b9feb54a798a6a08207b4d1177 (127.6.57.67:35373): tablet fff9b911a80c4393b8caf99a6beec9b0 (table test-table [id=800eaf2f96cd4691a22716efbfb6c321]) successfully deleted
I20260806 06:50:01.930711 26732 tablet_replica.cc:330] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: stopping tablet replica
I20260806 06:50:01.930817 26732 raft_consensus.cc:2229] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:01.930927 26732 raft_consensus.cc:2258] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:01.931350 26732 ts_tablet_manager.cc:1905] T fff9b911a80c4393b8caf99a6beec9b0 P bfc3299ed58746b580596151e41ce15a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 06:50:01.931962 26185 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.57.65:34641 (error 108)
I20260806 06:50:01.932193 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26410
I20260806 06:50:01.937177 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26551
I20260806 06:50:01.937254 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.937134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:34641 (local address 127.0.0.1:39752)
0806 06:50:01.937178 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.937219 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:34641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:50:01.937310 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:01.937229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:33377 (local address 127.0.0.1:53460)
0806 06:50:01.937268 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:01.937289 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.66:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 06:50:01.938423 26187 catalog_manager.cc:4584] TS bfc3299ed58746b580596151e41ce15a (127.6.57.66:33377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fff9b911a80c4393b8caf99a6beec9b0: Network error: Client connection negotiation failed: client connection to 127.6.57.66:33377: connect: Connection refused (error 111)
I20260806 06:50:01.942421 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26168
2026-08-06T06:50:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1946 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T06:50:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:01Z Disabled control of system clock
I20260806 06:50:01.971421 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:35079
--webserver_interface=127.6.57.126
--webserver_port=0
--builtin_ntp_servers=127.6.57.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:35079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:50:02.051892 26739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:02.052096 26739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:02.052117 26739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:02.053126 26739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:02.053162 26739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:02.053176 26739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:02.053189 26739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:02.053202 26739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:02.054397 26739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:35079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:35079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:02.054675 26739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:02.055022 26739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:02.059760 26745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.059724 26744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.059741 26747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:02.059938 26739 server_base.cc:1034] running on GCE node
I20260806 06:50:02.060316 26739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:02.060672 26739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:02.061858 26739 hybrid_clock.cc:648] HybridClock initialized: now 1785999002061834 us; error 37 us; skew 500 ppm
I20260806 06:50:02.063237 26739 webserver.cc:460] Webserver started at http://127.6.57.126:43625/ using document root <none> and password file <none>
I20260806 06:50:02.063498 26739 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:02.063573 26739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:02.063730 26739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:02.064741 26739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "b17c65be9ba64637bb04e638f97c79b2"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "d0bda8c8658bce0aa510e2657fa90e03"
server_key_iv: "d99f4a66e465c926629cb9e262e58468"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.065104 26739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "b17c65be9ba64637bb04e638f97c79b2"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "d0bda8c8658bce0aa510e2657fa90e03"
server_key_iv: "d99f4a66e465c926629cb9e262e58468"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.066360 26739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:02.067157 26753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.067330 26739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:02.067421 26739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "b17c65be9ba64637bb04e638f97c79b2"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "d0bda8c8658bce0aa510e2657fa90e03"
server_key_iv: "d99f4a66e465c926629cb9e262e58468"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.067500 26739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:02.088919 26739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:02.089236 26739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:02.089382 26739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:02.093425 26739 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:35079
I20260806 06:50:02.093456 26807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:35079 every 8 connection(s)
I20260806 06:50:02.093809 26739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:02.094862 26808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:02.096163 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26739
I20260806 06:50:02.096273 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:02.096480 6373 external_mini_cluster.cc:1428] Setting key fa9782e24fa1e4208f3ac84f55832429
I20260806 06:50:02.097504 26808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: Bootstrap starting.
I20260806 06:50:02.098229 26808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:02.098580 26808 log.cc:824] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:02.099291 26808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: No bootstrap required, opened a new log
I20260806 06:50:02.101397 26808 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } }
I20260806 06:50:02.101560 26808 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:02.101600 26808 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17c65be9ba64637bb04e638f97c79b2, State: Initialized, Role: FOLLOWER
I20260806 06:50:02.101755 26808 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } }
I20260806 06:50:02.101827 26808 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:02.101850 26808 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:02.101915 26808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:02.102447 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.096990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58178 (local address 127.6.57.126:35079)
0806 06:50:02.097214 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.097223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.097254 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:50:02.097796 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.097807 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.097897 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.098051 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:02.098242 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.099496 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.100216 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.100797 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.100953 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.101683 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:02.101688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:02.101688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:02.101910 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:02.101968 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:02.101984 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:02.102045 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.102180 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.102280 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260806 06:50:02.102639 26808 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } }
I20260806 06:50:02.102773 26808 leader_election.cc:302] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17c65be9ba64637bb04e638f97c79b2; no voters:
I20260806 06:50:02.102937 26808 leader_election.cc:288] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:02.102968 26813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:02.103075 26813 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [term 1 LEADER]: Becoming Leader. State: Replica: b17c65be9ba64637bb04e638f97c79b2, State: Running, Role: LEADER
I20260806 06:50:02.103160 26813 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } }
I20260806 06:50:02.103418 26808 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:02.103498 26813 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17c65be9ba64637bb04e638f97c79b2. Latest consensus state: current_term: 1 leader_uuid: "b17c65be9ba64637bb04e638f97c79b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } } }
I20260806 06:50:02.103510 26814 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17c65be9ba64637bb04e638f97c79b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17c65be9ba64637bb04e638f97c79b2" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 35079 } } }
I20260806 06:50:02.103582 26813 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:02.103614 26814 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:02.103763 26817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:02.104259 26817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:02.106011 26817 catalog_manager.cc:1347] Generated new cluster ID: ab66f8f07dc647969b057ee29a11d61a
I20260806 06:50:02.106069 26817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:02.124496 26817 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:02.124960 26817 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:02.126964 26817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: Generated new TSK 0
I20260806 06:50:02.127136 26817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:02.131165 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.65:0
--local_ip_for_outbound_sockets=127.6.57.65
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--builtin_ntp_servers=127.6.57.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:50:02.211179 26834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:02.211365 26834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:02.211390 26834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:02.212522 26834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:02.212561 26834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:02.212585 26834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.65
I20260806 06:50:02.213851 26834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.57.65
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.65
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:02.214097 26834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:02.214373 26834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:02.220613 26843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.220624 26840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.220664 26841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:02.220817 26834 server_base.cc:1034] running on GCE node
I20260806 06:50:02.221082 26834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:02.221273 26834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:02.222559 26834 hybrid_clock.cc:648] HybridClock initialized: now 1785999002222510 us; error 55 us; skew 500 ppm
I20260806 06:50:02.223863 26834 webserver.cc:460] Webserver started at http://127.6.57.65:34657/ using document root <none> and password file <none>
I20260806 06:50:02.224118 26834 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:02.224187 26834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:02.224344 26834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:02.225382 26834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data/instance:
uuid: "278909a4e2234712a44f384f124f3ab4"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "4a25a9cc953ee48eeddfcb93bb6dcf57"
server_key_iv: "536ce32c739e2ef01a7360d9ef183763"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.225741 26834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance:
uuid: "278909a4e2234712a44f384f124f3ab4"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "4a25a9cc953ee48eeddfcb93bb6dcf57"
server_key_iv: "536ce32c739e2ef01a7360d9ef183763"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.227185 26834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:02.227941 26849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.228108 26834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:02.228227 26834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal
uuid: "278909a4e2234712a44f384f124f3ab4"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "4a25a9cc953ee48eeddfcb93bb6dcf57"
server_key_iv: "536ce32c739e2ef01a7360d9ef183763"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.228309 26834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:02.240475 26834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:02.240749 26834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:02.240903 26834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:02.241122 26834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:02.241443 26834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:02.241496 26834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.241539 26834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:02.241586 26834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.247308 26834 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.65:39187
I20260806 06:50:02.247380 26968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.65:39187 every 8 connection(s)
I20260806 06:50:02.247695 26834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/data/info.pb
I20260806 06:50:02.252532 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.248504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.65:59183 (local address 127.6.57.126:35079)
0806 06:50:02.248590 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.248592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.248822 (+ 230us) server_negotiation.cc:407] Connection header received
0806 06:50:02.249296 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.249297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.249312 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.249367 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:02.249970 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.250458 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.251382 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.251549 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.251597 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.252321 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:02.252323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:02.252323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:02.252327 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:02.252339 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:02.252342 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:02.252381 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.252458 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.252477 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 06:50:02.252691 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.248492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:35079 (local address 127.6.57.65:59183)
0806 06:50:02.248708 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.248738 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.249203 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.249413 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.249416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.249424 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:02.249898 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.249902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.250587 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.250588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.251311 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.251313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.251618 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.251619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.251714 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.252204 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:50:02.252204 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:02.252228 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:50:02.252247 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:02.252392 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:02.252396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:02.252399 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.252444 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.252513 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
I20260806 06:50:02.253239 26969 heartbeater.cc:347] Connected to a master server at 127.6.57.126:35079
I20260806 06:50:02.253351 26969 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:02.253515 26969 heartbeater.cc:511] Master 127.6.57.126:35079 requested a full tablet report, sending...
I20260806 06:50:02.253998 26770 ts_manager.cc:194] Registered new tserver with Master: 278909a4e2234712a44f384f124f3ab4 (127.6.57.65:39187)
I20260806 06:50:02.254585 26770 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.65:59183
I20260806 06:50:02.256438 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26834
I20260806 06:50:02.256551 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-0/wal/instance
I20260806 06:50:02.256695 6373 external_mini_cluster.cc:1428] Setting key 600f83e6bf14cea4c7f5e1b99147e57d
I20260806 06:50:02.257459 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.66:0
--local_ip_for_outbound_sockets=127.6.57.66
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--builtin_ntp_servers=127.6.57.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:50:02.336814 26972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:02.337002 26972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:02.337025 26972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:02.338096 26972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:02.338130 26972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:02.338155 26972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.66
I20260806 06:50:02.339432 26972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.57.66
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.66
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:02.339607 26972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:02.339826 26972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:02.344636 26980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.344696 26978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.344712 26977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:02.344863 26972 server_base.cc:1034] running on GCE node
I20260806 06:50:02.345078 26972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:02.345377 26972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:02.346552 26972 hybrid_clock.cc:648] HybridClock initialized: now 1785999002346526 us; error 39 us; skew 500 ppm
I20260806 06:50:02.347733 26972 webserver.cc:460] Webserver started at http://127.6.57.66:42621/ using document root <none> and password file <none>
I20260806 06:50:02.347977 26972 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:02.348042 26972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:02.348174 26972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:02.349153 26972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data/instance:
uuid: "b2e1ed8bbda04519864efbf556eb40e5"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "fffed8a2ddcb415ef0793947da614849"
server_key_iv: "79b2d0c5326f1c3c3b9a6386bc5299ba"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.349493 26972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance:
uuid: "b2e1ed8bbda04519864efbf556eb40e5"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "fffed8a2ddcb415ef0793947da614849"
server_key_iv: "79b2d0c5326f1c3c3b9a6386bc5299ba"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.350775 26972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:02.351501 26987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.351672 26972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:02.351770 26972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal
uuid: "b2e1ed8bbda04519864efbf556eb40e5"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "fffed8a2ddcb415ef0793947da614849"
server_key_iv: "79b2d0c5326f1c3c3b9a6386bc5299ba"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.351850 26972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:02.359655 26972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:02.359917 26972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:02.360059 26972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:02.360283 26972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:02.360584 26972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:02.360661 26972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.360700 26972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:02.360728 26972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.366273 26972 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.66:42103
I20260806 06:50:02.366320 27107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.66:42103 every 8 connection(s)
I20260806 06:50:02.366688 26972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/data/info.pb
I20260806 06:50:02.371459 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.367363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.66:57797 (local address 127.6.57.126:35079)
0806 06:50:02.367412 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.367415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.367683 (+ 268us) server_negotiation.cc:407] Connection header received
0806 06:50:02.368073 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.368074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.368083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.368133 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:02.368715 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.369181 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.370202 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.370419 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.370533 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.371224 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:02.371225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:02.371226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:02.371229 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:02.371243 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:02.371246 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:02.371311 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.371384 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.371403 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:50:02.371670 27111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.367354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:35079 (local address 127.6.57.66:57797)
0806 06:50:02.367545 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.367575 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.367996 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.368187 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.368190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.368197 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:02.368649 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.368653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.369258 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.369258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.370100 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.370103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.370579 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.370580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.370693 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.371109 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:50:02.371109 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:02.371132 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:50:02.371151 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:02.371313 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:02.371316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:02.371318 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.371374 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.371441 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 06:50:02.372226 27108 heartbeater.cc:347] Connected to a master server at 127.6.57.126:35079
I20260806 06:50:02.372316 27108 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:02.372486 27108 heartbeater.cc:511] Master 127.6.57.126:35079 requested a full tablet report, sending...
I20260806 06:50:02.372605 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 26972
I20260806 06:50:02.372699 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal/instance
I20260806 06:50:02.372905 26770 ts_manager.cc:194] Registered new tserver with Master: b2e1ed8bbda04519864efbf556eb40e5 (127.6.57.66:42103)
I20260806 06:50:02.372910 6373 external_mini_cluster.cc:1428] Setting key d5d4f288f7e16b74da53136df04b6263
I20260806 06:50:02.373377 26770 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.66:57797
I20260806 06:50:02.374027 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.57.67:0
--local_ip_for_outbound_sockets=127.6.57.67
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--builtin_ntp_servers=127.6.57.84:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 06:50:02.453125 27112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:02.453308 27112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:02.453328 27112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:02.454427 27112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:02.454464 27112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:02.454509 27112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.57.67
I20260806 06:50:02.455822 27112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.57.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.57.67
--webserver_port=0
--tserver_master_addrs=127.6.57.126:35079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.57.67
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:02.456009 27112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:02.456225 27112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:02.461009 27121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.461022 27117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:02.461015 27119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:02.461222 27112 server_base.cc:1034] running on GCE node
I20260806 06:50:02.461491 27112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:02.461757 27112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:02.462951 27112 hybrid_clock.cc:648] HybridClock initialized: now 1785999002462947 us; error 34 us; skew 500 ppm
I20260806 06:50:02.464212 27112 webserver.cc:460] Webserver started at http://127.6.57.67:43199/ using document root <none> and password file <none>
I20260806 06:50:02.464452 27112 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:02.464521 27112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:02.464684 27112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:02.465698 27112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data/instance:
uuid: "e9e2ef674f964cf1b1d63b157bd903be"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "6c6e5142b9b451b89da02fe9f1436c2e"
server_key_iv: "b4d225dd8895f83e4d1864d01760ec05"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.466102 27112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance:
uuid: "e9e2ef674f964cf1b1d63b157bd903be"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "6c6e5142b9b451b89da02fe9f1436c2e"
server_key_iv: "b4d225dd8895f83e4d1864d01760ec05"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.467495 27112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:02.468251 27127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.468418 27112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:02.468525 27112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal
uuid: "e9e2ef674f964cf1b1d63b157bd903be"
format_stamp: "Formatted at 2026-08-06 06:50:02 on dist-test-slave-5l3k"
server_key: "6c6e5142b9b451b89da02fe9f1436c2e"
server_key_iv: "b4d225dd8895f83e4d1864d01760ec05"
server_key_version: "encryptionkey@0"
I20260806 06:50:02.468603 27112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:02.485157 27112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:02.485383 27112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:02.485476 27112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:02.485630 27112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 06:50:02.485954 27112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 06:50:02.486019 27112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.486059 27112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 06:50:02.486085 27112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:02.491838 27112 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.67:37115
I20260806 06:50:02.491914 27246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.67:37115 every 8 connection(s)
I20260806 06:50:02.492234 27112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/data/info.pb
I20260806 06:50:02.497259 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.492980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:43849 (local address 127.6.57.126:35079)
0806 06:50:02.493049 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.493052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.493325 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:50:02.493785 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.493786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.493795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.493850 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:02.494525 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.495009 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.495953 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.496136 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.496187 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.497043 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:02.497044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:02.497044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:02.497048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:02.497058 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:02.497061 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:02.497098 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.497182 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.497201 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 06:50:02.497481 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.492960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:35079 (local address 127.6.57.67:43849)
0806 06:50:02.493167 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.493211 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.493705 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.493907 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.493910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.493918 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:02.494415 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.494419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.495107 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.495108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.495877 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.495879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.496214 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.496214 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.496326 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.496915 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 06:50:02.496915 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:02.496943 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 06:50:02.496970 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:02.497114 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:02.497118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:02.497120 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.497173 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.497258 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 06:50:02.497948 27247 heartbeater.cc:347] Connected to a master server at 127.6.57.126:35079
I20260806 06:50:02.498039 27247 heartbeater.cc:464] Registering TS with master...
I20260806 06:50:02.498255 27247 heartbeater.cc:511] Master 127.6.57.126:35079 requested a full tablet report, sending...
I20260806 06:50:02.498653 26769 ts_manager.cc:194] Registered new tserver with Master: e9e2ef674f964cf1b1d63b157bd903be (127.6.57.67:37115)
I20260806 06:50:02.499027 26769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.57.67:43849
I20260806 06:50:02.499292 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27112
I20260806 06:50:02.499406 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal/instance
I20260806 06:50:02.499603 6373 external_mini_cluster.cc:1428] Setting key 46447b68939e7b92b78a05c3db694604
I20260806 06:50:02.500463 6373 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 06:50:02.504585 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.501802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58184 (local address 127.6.57.126:35079)
0806 06:50:02.501878 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.501880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.501920 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:50:02.501960 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.501961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.501969 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.502012 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:02.502170 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.502668 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.503381 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.503628 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.503681 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.504361 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:02.504362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:02.504363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:02.504367 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:02.504378 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:02.504381 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:02.504426 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.504463 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.504480 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 06:50:02.505939 26769 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 06:50:02.507578 26769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 06:50:02.513757 27256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.509718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39187 (local address 127.0.0.1:50060)
0806 06:50:02.510065 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.510075 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.510156 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.510542 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.510542 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.510553 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:02.510589 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.510591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511509 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.511511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.512705 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.512708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.513478 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.513554 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.513566 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.513636 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.513686 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":96,"threads_started":1}
I20260806 06:50:02.513768 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.509839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.6.57.65:39187)
0806 06:50:02.510144 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.510148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.510158 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:50:02.510296 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.510298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.510315 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.510394 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:02.510652 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511405 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.512793 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.513391 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.513572 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.513648 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.513686 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":83,"threads_started":1}
I20260806 06:50:02.514119 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.509936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.67:37115 (local address 127.0.0.1:37462)
0806 06:50:02.510162 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.510172 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.510212 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.510470 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.510471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.510472 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:02.510548 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.510550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511843 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.511844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.512893 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.512895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.513905 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.513968 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.513972 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.514011 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.514048 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":69,"threads_started":1}
I20260806 06:50:02.514391 27260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.510012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37462 (local address 127.6.57.67:37115)
0806 06:50:02.510331 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.510336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.510346 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:50:02.510377 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.510379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.510392 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.510457 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:02.510675 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511758 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.512973 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.513688 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513763 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.514247 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.514275 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.514312 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":73,"threads_started":1}
I20260806 06:50:02.514528 26900 tablet_service.cc:1467] Processing CreateTablet for tablet d1c1226e22cd45e78a374e9fb4e0eb01 (DEFAULT_TABLE table=test-table [id=b8622622375e4a7c838090afe24ac81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:02.514704 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.509801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.6.57.66:42103)
0806 06:50:02.510059 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.510064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.510125 (+ 61us) server_negotiation.cc:407] Connection header received
0806 06:50:02.510256 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.510258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.510273 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.510371 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:02.510785 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511865 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513077 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.513746 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.513802 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.514334 (+ 532us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.514573 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.514616 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260806 06:50:02.514926 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.509717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:42103 (local address 127.0.0.1:35660)
0806 06:50:02.510017 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.510056 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.510155 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.510381 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.510384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.510386 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:02.510690 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.510692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.511978 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.511979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.512939 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.512941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.514368 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.514369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.514434 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.514438 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.514808 (+ 370us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.514866 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":8,"thread_start_us":71,"threads_started":1}
I20260806 06:50:02.514959 26900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c1226e22cd45e78a374e9fb4e0eb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:02.515059 27178 tablet_service.cc:1467] Processing CreateTablet for tablet d1c1226e22cd45e78a374e9fb4e0eb01 (DEFAULT_TABLE table=test-table [id=b8622622375e4a7c838090afe24ac81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:02.515408 27178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c1226e22cd45e78a374e9fb4e0eb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:02.515553 27041 tablet_service.cc:1467] Processing CreateTablet for tablet d1c1226e22cd45e78a374e9fb4e0eb01 (DEFAULT_TABLE table=test-table [id=b8622622375e4a7c838090afe24ac81a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 06:50:02.515868 27041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c1226e22cd45e78a374e9fb4e0eb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:02.518002 27261 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: Bootstrap starting.
I20260806 06:50:02.518669 27261 tablet_bootstrap.cc:654] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:02.518816 27262 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Bootstrap starting.
I20260806 06:50:02.519008 27261 log.cc:824] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:02.519325 27263 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Bootstrap starting.
I20260806 06:50:02.519475 27262 tablet_bootstrap.cc:654] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:02.519769 27262 log.cc:824] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:02.519836 27261 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: No bootstrap required, opened a new log
I20260806 06:50:02.519910 27261 ts_tablet_manager.cc:1397] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 06:50:02.519949 27263 tablet_bootstrap.cc:654] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:02.520259 27263 log.cc:824] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:02.520578 27262 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: No bootstrap required, opened a new log
I20260806 06:50:02.520665 27262 ts_tablet_manager.cc:1397] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:02.521013 27263 tablet_bootstrap.cc:492] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: No bootstrap required, opened a new log
I20260806 06:50:02.521075 27263 ts_tablet_manager.cc:1397] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 06:50:02.522122 27261 raft_consensus.cc:348] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.522272 27261 raft_consensus.cc:374] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:02.522315 27261 raft_consensus.cc:729] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278909a4e2234712a44f384f124f3ab4, State: Initialized, Role: FOLLOWER
I20260806 06:50:02.522423 27261 consensus_queue.cc:260] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.522722 27261 ts_tablet_manager.cc:1428] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 06:50:02.522854 26969 heartbeater.cc:503] Master 127.6.57.126:35079 was elected leader, sending a full tablet report...
I20260806 06:50:02.523303 27262 raft_consensus.cc:348] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.523466 27262 raft_consensus.cc:374] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:02.523509 27262 raft_consensus.cc:729] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e2ef674f964cf1b1d63b157bd903be, State: Initialized, Role: FOLLOWER
I20260806 06:50:02.523633 27262 consensus_queue.cc:260] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.523576 27263 raft_consensus.cc:348] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.523705 27263 raft_consensus.cc:374] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:02.523741 27263 raft_consensus.cc:729] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e1ed8bbda04519864efbf556eb40e5, State: Initialized, Role: FOLLOWER
I20260806 06:50:02.523866 27262 ts_tablet_manager.cc:1428] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 06:50:02.523856 27263 consensus_queue.cc:260] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.523910 27247 heartbeater.cc:503] Master 127.6.57.126:35079 was elected leader, sending a full tablet report...
I20260806 06:50:02.524096 27263 ts_tablet_manager.cc:1428] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 06:50:02.524230 27108 heartbeater.cc:503] Master 127.6.57.126:35079 was elected leader, sending a full tablet report...
W20260806 06:50:02.617210 27109 tablet.cc:2369] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:02.709635 27268 raft_consensus.cc:482] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 06:50:02.709784 27268 raft_consensus.cc:504] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.710214 27268 leader_election.cc:288] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2e1ed8bbda04519864efbf556eb40e5 (127.6.57.66:42103), 278909a4e2234712a44f384f124f3ab4 (127.6.57.65:39187)
I20260806 06:50:02.713313 27258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.710528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:38477 (local address 127.6.57.65:39187)
0806 06:50:02.710585 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.710588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.710599 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:02.710633 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.710634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.710646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.710712 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:02.710891 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.711347 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.712460 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.712970 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.713022 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.713030 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.713226 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.713259 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:50:02.713320 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.710420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39187 (local address 127.6.57.67:38477)
0806 06:50:02.710474 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.710486 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.710546 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.710743 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.710745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.710746 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:02.710821 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.710823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.711429 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.711430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.712365 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.712368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.713075 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.713076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.713150 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.713154 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.713210 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.713249 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 06:50:02.713735 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.710541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.57.67:44767 (local address 127.6.57.66:42103)
0806 06:50:02.710585 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.710588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.710881 (+ 293us) server_negotiation.cc:407] Connection header received
0806 06:50:02.710908 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.710909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.710917 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.710973 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 06:50:02.711153 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.711768 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.712844 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.713326 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.713391 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.713402 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.713657 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.713690 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:02.713701 26922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c1226e22cd45e78a374e9fb4e0eb01" candidate_uuid: "e9e2ef674f964cf1b1d63b157bd903be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278909a4e2234712a44f384f124f3ab4" is_pre_election: true
I20260806 06:50:02.713765 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.710428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.66:42103 (local address 127.6.57.67:44767)
0806 06:50:02.710657 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.710664 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:02.710725 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:02.710997 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:02.710998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:02.710999 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 06:50:02.711070 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.711071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.711885 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.711885 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.712753 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:02.712756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.713495 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.713497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:02.713573 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.713576 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 06:50:02.713652 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 06:50:02.713685 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":23,"thread_start_us":140,"threads_started":1}
I20260806 06:50:02.713896 26922 raft_consensus.cc:2454] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e2ef674f964cf1b1d63b157bd903be in term 0.
I20260806 06:50:02.714210 27131 leader_election.cc:302] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 278909a4e2234712a44f384f124f3ab4, e9e2ef674f964cf1b1d63b157bd903be; no voters:
I20260806 06:50:02.714268 27061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c1226e22cd45e78a374e9fb4e0eb01" candidate_uuid: "e9e2ef674f964cf1b1d63b157bd903be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e1ed8bbda04519864efbf556eb40e5" is_pre_election: true
I20260806 06:50:02.714358 27268 raft_consensus.cc:2781] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 06:50:02.714399 27061 raft_consensus.cc:2454] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9e2ef674f964cf1b1d63b157bd903be in term 0.
I20260806 06:50:02.714439 27268 raft_consensus.cc:482] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 06:50:02.714468 27268 raft_consensus.cc:3037] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:02.715270 27268 raft_consensus.cc:504] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.715402 27268 leader_election.cc:288] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [CANDIDATE]: Term 1 election: Requested vote from peers b2e1ed8bbda04519864efbf556eb40e5 (127.6.57.66:42103), 278909a4e2234712a44f384f124f3ab4 (127.6.57.65:39187)
I20260806 06:50:02.715538 27061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c1226e22cd45e78a374e9fb4e0eb01" candidate_uuid: "e9e2ef674f964cf1b1d63b157bd903be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2e1ed8bbda04519864efbf556eb40e5"
I20260806 06:50:02.715598 27061 raft_consensus.cc:3037] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:02.716364 27061 raft_consensus.cc:2454] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e2ef674f964cf1b1d63b157bd903be in term 1.
I20260806 06:50:02.716477 26922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c1226e22cd45e78a374e9fb4e0eb01" candidate_uuid: "e9e2ef674f964cf1b1d63b157bd903be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "278909a4e2234712a44f384f124f3ab4"
I20260806 06:50:02.716542 27128 leader_election.cc:302] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2e1ed8bbda04519864efbf556eb40e5, e9e2ef674f964cf1b1d63b157bd903be; no voters:
I20260806 06:50:02.716565 26922 raft_consensus.cc:3037] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:02.716677 27268 raft_consensus.cc:2781] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:02.716825 27268 raft_consensus.cc:686] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 1 LEADER]: Becoming Leader. State: Replica: e9e2ef674f964cf1b1d63b157bd903be, State: Running, Role: LEADER
I20260806 06:50:02.716936 27268 consensus_queue.cc:237] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } }
I20260806 06:50:02.717523 26922 raft_consensus.cc:2454] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9e2ef674f964cf1b1d63b157bd903be in term 1.
I20260806 06:50:02.717866 26769 catalog_manager.cc:5495] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be reported cstate change: term changed from 0 to 1, leader changed from <none> to e9e2ef674f964cf1b1d63b157bd903be (127.6.57.67). New cstate: current_term: 1 leader_uuid: "e9e2ef674f964cf1b1d63b157bd903be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9e2ef674f964cf1b1d63b157bd903be" member_type: VOTER last_known_addr { host: "127.6.57.67" port: 37115 } health_report { overall_health: HEALTHY } } }
I20260806 06:50:02.738826 27260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.736679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37470 (local address 127.6.57.67:37115)
0806 06:50:02.736740 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:02.736742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:02.736750 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:02.736781 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:02.736782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:02.736798 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:02.736865 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:02.736962 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.737436 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.738148 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:02.738318 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:02.738371 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:02.738551 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:02.738641 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:02.738709 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:02.738744 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 06:50:02.738759 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 06:50:02.740640 27061 raft_consensus.cc:1261] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 1 FOLLOWER]: Refusing update from remote peer e9e2ef674f964cf1b1d63b157bd903be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:02.740720 26922 raft_consensus.cc:1261] T d1c1226e22cd45e78a374e9fb4e0eb01 P 278909a4e2234712a44f384f124f3ab4 [term 1 FOLLOWER]: Refusing update from remote peer e9e2ef674f964cf1b1d63b157bd903be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:02.740985 27268 consensus_queue.cc:1035] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2e1ed8bbda04519864efbf556eb40e5" member_type: VOTER last_known_addr { host: "127.6.57.66" port: 42103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:02.741127 27271 consensus_queue.cc:1035] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [LEADER]: Connected to new peer: Peer: permanent_uuid: "278909a4e2234712a44f384f124f3ab4" member_type: VOTER last_known_addr { host: "127.6.57.65" port: 39187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:50:02.742719 27248 tablet.cc:2369] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 06:50:02.744161 27275 mvcc.cc:205] Tried to move back new op lower bound from 7315451915222892544 to 7315451915129212928. Current Snapshot: MvccSnapshot[applied={T|T < 7315451915222892544 or (T in {7315451915222892544})}]
I20260806 06:50:02.745729 26770 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:02.745819 26770 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 06:50:02.746690 26770 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b17c65be9ba64637bb04e638f97c79b2: Sending DeleteTablet for 3 replicas of tablet d1c1226e22cd45e78a374e9fb4e0eb01
I20260806 06:50:02.747123 27178 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c1226e22cd45e78a374e9fb4e0eb01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:02 UTC) from {username='slave'} at 127.0.0.1:37462
I20260806 06:50:02.747242 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26834
I20260806 06:50:02.747393 27041 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c1226e22cd45e78a374e9fb4e0eb01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 06:50:02 UTC) from {username='slave'} at 127.0.0.1:35660
I20260806 06:50:02.748755 27285 tablet_replica.cc:330] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: stopping tablet replica
I20260806 06:50:02.748883 27285 raft_consensus.cc:2229] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 06:50:02.748993 27285 raft_consensus.cc:2258] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:02.749166 27284 tablet_replica.cc:330] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: stopping tablet replica
I20260806 06:50:02.749260 27284 raft_consensus.cc:2229] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 1 LEADER]: Raft consensus shutting down.
I20260806 06:50:02.749359 27284 raft_consensus.cc:2258] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 06:50:02.749630 27285 ts_tablet_manager.cc:1905] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:02.749796 27284 ts_tablet_manager.cc:1905] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 06:50:02.751516 27285 ts_tablet_manager.cc:1918] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:02.751518 27284 ts_tablet_manager.cc:1918] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 06:50:02.751598 27284 log.cc:1196] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-2/wal/wals/d1c1226e22cd45e78a374e9fb4e0eb01
I20260806 06:50:02.751721 27285 log.cc:1196] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Deleting WAL directory at /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785998763187424-6373-0/minicluster-data/ts-1/wal/wals/d1c1226e22cd45e78a374e9fb4e0eb01
I20260806 06:50:02.751902 27284 ts_tablet_manager.cc:1939] T d1c1226e22cd45e78a374e9fb4e0eb01 P e9e2ef674f964cf1b1d63b157bd903be: Deleting consensus metadata
I20260806 06:50:02.752159 27285 ts_tablet_manager.cc:1939] T d1c1226e22cd45e78a374e9fb4e0eb01 P b2e1ed8bbda04519864efbf556eb40e5: Deleting consensus metadata
I20260806 06:50:02.752329 26757 catalog_manager.cc:4841] TS e9e2ef674f964cf1b1d63b157bd903be (127.6.57.67:37115): tablet d1c1226e22cd45e78a374e9fb4e0eb01 (table test-table [id=b8622622375e4a7c838090afe24ac81a]) successfully deleted
I20260806 06:50:02.752704 26754 catalog_manager.cc:4841] TS b2e1ed8bbda04519864efbf556eb40e5 (127.6.57.66:42103): tablet d1c1226e22cd45e78a374e9fb4e0eb01 (table test-table [id=b8622622375e4a7c838090afe24ac81a]) successfully deleted
W20260806 06:50:02.755841 26757 connection.cc:535] client connection to 127.6.57.65:39187 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 06:50:02.755955 26757 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 06:50:02.756147 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26972
I20260806 06:50:02.756402 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:02.756283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.65:39187 (local address 127.0.0.1:50064)
0806 06:50:02.756330 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:02.756366 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:50:02.756495 26757 catalog_manager.cc:4584] TS 278909a4e2234712a44f384f124f3ab4 (127.6.57.65:39187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1c1226e22cd45e78a374e9fb4e0eb01: Network error: Client connection negotiation failed: client connection to 127.6.57.65:39187: connect: Connection refused (error 111)
I20260806 06:50:02.762385 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27112
I20260806 06:50:02.768615 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 26739
2026-08-06T06:50:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (825 ms)
[----------] 6 tests from EncryptionPolicyTest (8050 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:02 dist-test-slave-5l3k krb5kdc[27289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:02 dist-test-slave-5l3k krb5kdc[27289](info): set up 2 sockets
Aug 06 06:50:02 dist-test-slave-5l3k krb5kdc[27289](info): commencing operation
krb5kdc: starting...
W20260806 06:50:03.812798 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:03 dist-test-slave-5l3k krb5kdc[27289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:03.929626 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:04.009702 27302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:04.009903 27302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:04.009927 27302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:04.011030 27302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:04.011065 27302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:04.011080 27302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:04.011091 27302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:04.011103 27302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:04.012502 27302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:04.012694 27302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:04.012863 27302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:04.017654 27308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:04.017649 27309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:04.017899 27302 server_base.cc:1034] running on GCE node
W20260806 06:50:04.017655 27311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:04.018323 27302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:04.018635 27302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:04.019795 27302 hybrid_clock.cc:648] HybridClock initialized: now 1785999004019785 us; error 34 us; skew 500 ppm
Aug 06 06:50:04 dist-test-slave-5l3k krb5kdc[27289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:04.022239 27302 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:04.022681 27302 webserver.cc:460] Webserver started at http://127.0.0.1:38627/ using document root <none> and password file <none>
I20260806 06:50:04.022867 27302 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:04.022907 27302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:04.022992 27302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:04.023974 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "d140089a5aff434cba167424508373ff"
format_stamp: "Formatted at 2026-08-06 06:50:04 on dist-test-slave-5l3k"
server_key: "4760703f03e35e1c0903f5ffcd1b6b38"
server_key_iv: "fc8b673d7485ee5aa5eb9d2cf1dc55f3"
server_key_version: "encryptionkey@0"
I20260806 06:50:04.024268 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "d140089a5aff434cba167424508373ff"
format_stamp: "Formatted at 2026-08-06 06:50:04 on dist-test-slave-5l3k"
server_key: "4760703f03e35e1c0903f5ffcd1b6b38"
server_key_iv: "fc8b673d7485ee5aa5eb9d2cf1dc55f3"
server_key_version: "encryptionkey@0"
I20260806 06:50:04.025501 27302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:04.026146 27318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:04.026312 27302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:04.026408 27302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "d140089a5aff434cba167424508373ff"
format_stamp: "Formatted at 2026-08-06 06:50:04 on dist-test-slave-5l3k"
server_key: "4760703f03e35e1c0903f5ffcd1b6b38"
server_key_iv: "fc8b673d7485ee5aa5eb9d2cf1dc55f3"
server_key_version: "encryptionkey@0"
I20260806 06:50:04.026499 27302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:04.037878 27302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:04.039719 27302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:04.039871 27302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:04.043965 27302 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41415
I20260806 06:50:04.044023 27374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41415 every 8 connection(s)
I20260806 06:50:04.044341 27302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:04.045248 27375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:04.047466 27375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff: Bootstrap starting.
I20260806 06:50:04.048044 27375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:04.048339 27375 log.cc:824] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:04.049062 27375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff: No bootstrap required, opened a new log
I20260806 06:50:04.050626 27375 raft_consensus.cc:348] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } }
I20260806 06:50:04.050746 27375 raft_consensus.cc:374] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:04.050791 27375 raft_consensus.cc:729] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d140089a5aff434cba167424508373ff, State: Initialized, Role: FOLLOWER
I20260806 06:50:04.050930 27375 consensus_queue.cc:260] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } }
I20260806 06:50:04.050987 27375 raft_consensus.cc:388] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:04.051029 27375 raft_consensus.cc:482] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:04.051097 27375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:04.051625 27375 raft_consensus.cc:504] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } }
I20260806 06:50:04.051734 27375 leader_election.cc:302] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d140089a5aff434cba167424508373ff; no voters:
I20260806 06:50:04.051908 27375 leader_election.cc:288] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:04.051945 27378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:04.052093 27378 raft_consensus.cc:686] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [term 1 LEADER]: Becoming Leader. State: Replica: d140089a5aff434cba167424508373ff, State: Running, Role: LEADER
I20260806 06:50:04.052157 27375 sys_catalog.cc:564] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:04.052181 27378 consensus_queue.cc:237] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } }
I20260806 06:50:04.052554 27379 sys_catalog.cc:455] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d140089a5aff434cba167424508373ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } } }
I20260806 06:50:04.052639 27379 sys_catalog.cc:458] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:04.052791 27380 sys_catalog.cc:455] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d140089a5aff434cba167424508373ff. Latest consensus state: current_term: 1 leader_uuid: "d140089a5aff434cba167424508373ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d140089a5aff434cba167424508373ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41415 } } }
I20260806 06:50:04.052930 27380 sys_catalog.cc:458] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:04.053078 27384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:04.053627 27384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:04.054116 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27302
I20260806 06:50:04.054327 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:04.054594 6373 external_mini_cluster.cc:1428] Setting key 6d4a5a1529c974362329dfd5e7314112
I20260806 06:50:04.055338 27384 catalog_manager.cc:1347] Generated new cluster ID: 5e0da60261024e2499705bbc024ad6e7
I20260806 06:50:04.055384 27384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:04 dist-test-slave-5l3k krb5kdc[27289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:50:04.063474 27384 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:04.064108 27384 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:04.066129 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:04.055060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46833 (local address 127.0.0.1:41415)
0806 06:50:04.055203 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:04.055212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:04.055554 (+ 342us) server_negotiation.cc:407] Connection header received
0806 06:50:04.055701 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:04.055708 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:04.055768 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:04.055884 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:04.056675 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:04.058067 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:04.059191 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:04.059775 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:04.059868 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:04.062706 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:04.062710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:04.062710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:04.062722 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:04.064594 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:04.064974 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:04.064977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:04.064978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:04.065070 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:04.065300 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:04.065301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:04.065302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:04.065521 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:04.065712 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:04.065794 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:50:04.065895 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260806 06:50:04.067051 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:04.067680 27384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d140089a5aff434cba167424508373ff: Generated new TSK 0
I20260806 06:50:04.067840 27384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:04.074538 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:04.068523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38085 (local address 127.0.0.1:41415)
0806 06:50:04.068592 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:04.068594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:04.068712 (+ 118us) server_negotiation.cc:407] Connection header received
0806 06:50:04.068781 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:04.068782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:04.068791 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:04.068832 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:04.069323 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:04.069802 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:04.070515 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:04.070736 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:04.070812 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:04.071931 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:04.071932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:04.071933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:04.071939 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:04.073313 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:04.073740 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:04.073741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:04.073741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:04.073794 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:04.074023 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:04.074024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:04.074024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:04.074130 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:04.074252 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:04.074368 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 06:50:04.074428 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 06:50:04.076308 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27302
W20260806 06:50:04.081902 27287 connection.cc:535] client connection to 127.0.0.1:41415 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41415): Connection reset by peer (error 104)
2026-08-06T06:50:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:04 dist-test-slave-5l3k krb5kdc[27408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:04 dist-test-slave-5l3k krb5kdc[27408](info): set up 2 sockets
Aug 06 06:50:04 dist-test-slave-5l3k krb5kdc[27408](info): commencing operation
krb5kdc: starting...
W20260806 06:50:05.138562 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:05 dist-test-slave-5l3k krb5kdc[27408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:05.253121 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:05.332857 27421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:05.333047 27421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:05.333068 27421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:05.334097 27421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:05.334131 27421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:05.334144 27421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:05.334157 27421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:05.334168 27421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:05.335539 27421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:05.335752 27421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:05.335979 27421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:05.340865 27429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:05.340829 27431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:05.340880 27421 server_base.cc:1034] running on GCE node
W20260806 06:50:05.340849 27428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:05.341409 27421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:05.341694 27421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:05.342859 27421 hybrid_clock.cc:648] HybridClock initialized: now 1785999005342848 us; error 45 us; skew 500 ppm
Aug 06 06:50:05 dist-test-slave-5l3k krb5kdc[27408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:05.350840 27421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:05.351254 27421 webserver.cc:460] Webserver started at http://127.0.0.1:45675/ using document root <none> and password file <none>
I20260806 06:50:05.351490 27421 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:05.351557 27421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:05.351713 27421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:05.352731 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "38f3b98a730743e6aac3c3a9fe5ced9b"
format_stamp: "Formatted at 2026-08-06 06:50:05 on dist-test-slave-5l3k"
server_key: "3e10a838c32e8b13dc1c39450820548a"
server_key_iv: "130c4a890577eb051c692424adda5932"
server_key_version: "encryptionkey@0"
I20260806 06:50:05.353075 27421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "38f3b98a730743e6aac3c3a9fe5ced9b"
format_stamp: "Formatted at 2026-08-06 06:50:05 on dist-test-slave-5l3k"
server_key: "3e10a838c32e8b13dc1c39450820548a"
server_key_iv: "130c4a890577eb051c692424adda5932"
server_key_version: "encryptionkey@0"
I20260806 06:50:05.354334 27421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:05.355165 27438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:05.355355 27421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:05.355454 27421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "38f3b98a730743e6aac3c3a9fe5ced9b"
format_stamp: "Formatted at 2026-08-06 06:50:05 on dist-test-slave-5l3k"
server_key: "3e10a838c32e8b13dc1c39450820548a"
server_key_iv: "130c4a890577eb051c692424adda5932"
server_key_version: "encryptionkey@0"
I20260806 06:50:05.355541 27421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:05.369585 27421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:05.371492 27421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:05.371646 27421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:05.375566 27421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34079
I20260806 06:50:05.375615 27492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34079 every 8 connection(s)
I20260806 06:50:05.375957 27421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:05.376955 27493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:05.377658 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27421
I20260806 06:50:05.377797 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:05.377986 6373 external_mini_cluster.cc:1428] Setting key 143a8212e904a139f636136f220a7ea0
I20260806 06:50:05.380206 27493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b: Bootstrap starting.
I20260806 06:50:05.380791 27493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:05.381083 27493 log.cc:824] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:05.381755 27493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b: No bootstrap required, opened a new log
I20260806 06:50:05.383422 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:05.378501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33805 (local address 127.0.0.1:34079)
0806 06:50:05.378698 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:05.378706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:05.378741 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:50:05.379291 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:05.379301 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:05.379376 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:05.379516 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:05.379860 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:05.380801 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:05.381638 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:05.382085 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:05.382198 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:05.382871 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:05.382875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:05.382876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:05.383046 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:05.383072 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:05.383082 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:05.383172 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:05.383244 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 06:50:05.383270 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260806 06:50:05.383795 27493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } }
I20260806 06:50:05.383944 27493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:05.383980 27493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f3b98a730743e6aac3c3a9fe5ced9b, State: Initialized, Role: FOLLOWER
I20260806 06:50:05.384121 27493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } }
I20260806 06:50:05.384224 27493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:05.384267 27493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:05.384316 27493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:05.384357 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:05.384841 27493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } }
I20260806 06:50:05.384980 27493 leader_election.cc:302] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38f3b98a730743e6aac3c3a9fe5ced9b; no voters:
I20260806 06:50:05.385164 27493 leader_election.cc:288] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:05.385192 27499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:05.385304 27499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [term 1 LEADER]: Becoming Leader. State: Replica: 38f3b98a730743e6aac3c3a9fe5ced9b, State: Running, Role: LEADER
I20260806 06:50:05.385381 27499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } }
I20260806 06:50:05.385411 27493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:05.385713 27499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } } }
I20260806 06:50:05.385787 27499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:05.385912 27499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f3b98a730743e6aac3c3a9fe5ced9b. Latest consensus state: current_term: 1 leader_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f3b98a730743e6aac3c3a9fe5ced9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34079 } } }
I20260806 06:50:05.385980 27499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:05.386133 27509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:05.387138 27509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:05.388803 27509 catalog_manager.cc:1347] Generated new cluster ID: 91ce77151a744056a9db264f8270ae58
I20260806 06:50:05.388856 27509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:05.390213 27497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:05.385761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41289 (local address 127.0.0.1:34079)
0806 06:50:05.385794 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:05.385796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:05.386752 (+ 956us) server_negotiation.cc:407] Connection header received
0806 06:50:05.387123 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:05.387124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:05.387135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:05.387192 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:05.387442 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:05.388070 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:05.388995 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:05.389215 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:05.389307 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:05.389996 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:05.389997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:05.389998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:05.390002 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:05.390014 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:05.390018 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:05.390070 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:05.390132 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:50:05.390141 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:05.399963 27509 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:05.400529 27509 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:05.405417 27509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38f3b98a730743e6aac3c3a9fe5ced9b: Generated new TSK 0
I20260806 06:50:05.405599 27509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:05.408617 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27421
W20260806 06:50:05.414186 27406 connection.cc:535] client connection to 127.0.0.1:34079 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34079): Connection reset by peer (error 104)
2026-08-06T06:50:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:05 dist-test-slave-5l3k krb5kdc[27528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:05 dist-test-slave-5l3k krb5kdc[27528](info): set up 2 sockets
Aug 06 06:50:05 dist-test-slave-5l3k krb5kdc[27528](info): commencing operation
krb5kdc: starting...
W20260806 06:50:06.452433 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:06 dist-test-slave-5l3k krb5kdc[27528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:06.571529 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:06.654189 27542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:06.654376 27542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:06.654397 27542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:06.655598 27542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:06.655642 27542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:06.655656 27542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:06.655668 27542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:06.655718 27542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:06.657203 27542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:06.657442 27542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:06.657672 27542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:06.662508 27550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:06.662544 27542 server_base.cc:1034] running on GCE node
W20260806 06:50:06.662570 27548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:06.662516 27547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:06.662946 27542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:06.663309 27542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:06.664501 27542 hybrid_clock.cc:648] HybridClock initialized: now 1785999006664485 us; error 45 us; skew 500 ppm
Aug 06 06:50:06 dist-test-slave-5l3k krb5kdc[27528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:06.667140 27542 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:06.667605 27542 webserver.cc:460] Webserver started at http://127.0.0.1:43183/ using document root <none> and password file <none>
I20260806 06:50:06.667858 27542 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:06.667929 27542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:06.668082 27542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:06.669080 27542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "183f121333bd4c6c9d57816268d21506"
format_stamp: "Formatted at 2026-08-06 06:50:06 on dist-test-slave-5l3k"
server_key: "2c82c2c335c20bc175e7d3b9577dd266"
server_key_iv: "91cc18be37a6c738176f4805f7976855"
server_key_version: "encryptionkey@0"
I20260806 06:50:06.669437 27542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "183f121333bd4c6c9d57816268d21506"
format_stamp: "Formatted at 2026-08-06 06:50:06 on dist-test-slave-5l3k"
server_key: "2c82c2c335c20bc175e7d3b9577dd266"
server_key_iv: "91cc18be37a6c738176f4805f7976855"
server_key_version: "encryptionkey@0"
I20260806 06:50:06.670816 27542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:06.671531 27557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:06.671706 27542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:06.671813 27542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "183f121333bd4c6c9d57816268d21506"
format_stamp: "Formatted at 2026-08-06 06:50:06 on dist-test-slave-5l3k"
server_key: "2c82c2c335c20bc175e7d3b9577dd266"
server_key_iv: "91cc18be37a6c738176f4805f7976855"
server_key_version: "encryptionkey@0"
I20260806 06:50:06.671885 27542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:06.675143 27542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:06.677386 27542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:06.677544 27542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:06.681620 27542 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34725
I20260806 06:50:06.681653 27611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34725 every 8 connection(s)
I20260806 06:50:06.681954 27542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:06.682964 27612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:06.685134 27612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506: Bootstrap starting.
I20260806 06:50:06.685722 27612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:06.685997 27612 log.cc:824] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:06.686302 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27542
I20260806 06:50:06.686431 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:06.686741 27612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506: No bootstrap required, opened a new log
I20260806 06:50:06.686753 6373 external_mini_cluster.cc:1428] Setting key 06a8e8e91fe821eb5fcdf9937d57f84c
I20260806 06:50:06.688598 27612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } }
I20260806 06:50:06.688772 27612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:06.688812 27612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183f121333bd4c6c9d57816268d21506, State: Initialized, Role: FOLLOWER
I20260806 06:50:06.688999 27612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } }
I20260806 06:50:06.689107 27612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:06.689148 27612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:06.689203 27612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:06.689924 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:06.687318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42509 (local address 127.0.0.1:34725)
0806 06:50:06.687547 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:06.687554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:06.687579 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:50:06.688606 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:06.688614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:06.688644 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:06.688762 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:06.689439 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:06.689442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:06.689442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:06.689446 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:06.689481 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:06.689504 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:06.689559 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:06.689660 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 06:50:06.689691 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":108,"threads_started":1}
I20260806 06:50:06.690342 27612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } }
I20260806 06:50:06.690451 27612 leader_election.cc:302] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 183f121333bd4c6c9d57816268d21506; no voters:
I20260806 06:50:06.690650 27612 leader_election.cc:288] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:06.690677 27617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:06.690788 27617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [term 1 LEADER]: Becoming Leader. State: Replica: 183f121333bd4c6c9d57816268d21506, State: Running, Role: LEADER
I20260806 06:50:06.690903 27612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:06.690923 27617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } }
I20260806 06:50:06.690980 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:06.691318 27618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "183f121333bd4c6c9d57816268d21506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } } }
I20260806 06:50:06.691614 27618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:06.691529 27619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 183f121333bd4c6c9d57816268d21506. Latest consensus state: current_term: 1 leader_uuid: "183f121333bd4c6c9d57816268d21506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "183f121333bd4c6c9d57816268d21506" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34725 } } }
I20260806 06:50:06.691812 27619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:06.691946 27626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:06.692548 27626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:06.694830 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:06.693519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54079 (local address 127.0.0.1:34725)
0806 06:50:06.693555 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:06.693557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:06.693830 (+ 273us) server_negotiation.cc:407] Connection header received
0806 06:50:06.693882 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:06.693883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:06.693889 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:06.693967 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:06.694439 (+ 472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:06.694440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:06.694440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:06.694444 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:06.694456 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:06.694460 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:06.694675 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:06.694731 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 06:50:06.694747 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:06.695719 27626 catalog_manager.cc:1347] Generated new cluster ID: c3bbbf2226604a9a8c0c3e6d4b84941d
I20260806 06:50:06.695782 27626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:06.704773 27626 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:06.705003 27626 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:06.716657 27626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 183f121333bd4c6c9d57816268d21506: Generated new TSK 0
I20260806 06:50:06.716856 27626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:06.724690 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27542
2026-08-06T06:50:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:06 dist-test-slave-5l3k krb5kdc[27645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:06 dist-test-slave-5l3k krb5kdc[27645](info): set up 2 sockets
Aug 06 06:50:06 dist-test-slave-5l3k krb5kdc[27645](info): commencing operation
krb5kdc: starting...
W20260806 06:50:07.776618 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:07 dist-test-slave-5l3k krb5kdc[27645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:07.895555 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:07.980561 27659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:07.980748 27659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:07.980770 27659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:07.981861 27659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:07.981900 27659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:07.981914 27659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:07.981926 27659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:07.981976 27659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:07.983428 27659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:07.983680 27659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:07.983897 27659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:07.989264 27665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:07.989260 27667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:07.989265 27664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:07.989377 27659 server_base.cc:1034] running on GCE node
I20260806 06:50:07.989781 27659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:07.990094 27659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:07.991276 27659 hybrid_clock.cc:648] HybridClock initialized: now 1785999007991269 us; error 37 us; skew 500 ppm
Aug 06 06:50:07 dist-test-slave-5l3k krb5kdc[27645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999007, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:07.993726 27659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:07.994169 27659 webserver.cc:460] Webserver started at http://127.0.0.1:37033/ using document root <none> and password file <none>
I20260806 06:50:07.994381 27659 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:07.994419 27659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:07.994558 27659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:07.995525 27659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "73f7216b04a54be48169db1f7638f261"
format_stamp: "Formatted at 2026-08-06 06:50:07 on dist-test-slave-5l3k"
server_key: "7368beaa0fcd0ea03e334724af7154d4"
server_key_iv: "938582ea284a573275173f97ad39e767"
server_key_version: "encryptionkey@0"
I20260806 06:50:07.995805 27659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "73f7216b04a54be48169db1f7638f261"
format_stamp: "Formatted at 2026-08-06 06:50:07 on dist-test-slave-5l3k"
server_key: "7368beaa0fcd0ea03e334724af7154d4"
server_key_iv: "938582ea284a573275173f97ad39e767"
server_key_version: "encryptionkey@0"
I20260806 06:50:07.997087 27659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:07.997751 27675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:07.997937 27659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:07.998013 27659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "73f7216b04a54be48169db1f7638f261"
format_stamp: "Formatted at 2026-08-06 06:50:07 on dist-test-slave-5l3k"
server_key: "7368beaa0fcd0ea03e334724af7154d4"
server_key_iv: "938582ea284a573275173f97ad39e767"
server_key_version: "encryptionkey@0"
I20260806 06:50:07.998067 27659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:08.009305 27659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:08.011263 27659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:08.011379 27659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:08.015146 27659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43177
I20260806 06:50:08.015196 27728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43177 every 8 connection(s)
I20260806 06:50:08.015503 27659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:08.016481 27729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:08.018785 27729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261: Bootstrap starting.
I20260806 06:50:08.019353 27729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:08.019647 27729 log.cc:824] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:08.020299 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27659
I20260806 06:50:08.020391 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:08.020448 27729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261: No bootstrap required, opened a new log
I20260806 06:50:08.020632 6373 external_mini_cluster.cc:1428] Setting key 5942948025e7248a14196d0e855b7efe
I20260806 06:50:08.022207 27729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } }
I20260806 06:50:08.022358 27729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:08.022423 27729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f7216b04a54be48169db1f7638f261, State: Initialized, Role: FOLLOWER
I20260806 06:50:08.022619 27729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } }
I20260806 06:50:08.022718 27729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:08.022758 27729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:08.022809 27729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:08.023607 27729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } }
I20260806 06:50:08.023727 27729 leader_election.cc:302] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73f7216b04a54be48169db1f7638f261; no voters:
I20260806 06:50:08.023910 27729 leader_election.cc:288] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:08.023954 27734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:08.024108 27734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [term 1 LEADER]: Becoming Leader. State: Replica: 73f7216b04a54be48169db1f7638f261, State: Running, Role: LEADER
I20260806 06:50:08.024196 27729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:08.024268 27734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } }
I20260806 06:50:08.024657 27735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73f7216b04a54be48169db1f7638f261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } } }
I20260806 06:50:08.024845 27735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:08.024647 27734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73f7216b04a54be48169db1f7638f261. Latest consensus state: current_term: 1 leader_uuid: "73f7216b04a54be48169db1f7638f261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f7216b04a54be48169db1f7638f261" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43177 } } }
I20260806 06:50:08.025357 27734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:08.025592 27743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:08.026182 27743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:08.028003 27743 catalog_manager.cc:1347] Generated new cluster ID: 31bdb05aa2194ad48d39aad99ba110e4
I20260806 06:50:08.028060 27743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:08 dist-test-slave-5l3k krb5kdc[27645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:50:08.032800 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:08.021157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34353 (local address 127.0.0.1:43177)
0806 06:50:08.021341 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:08.021348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:08.021372 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:50:08.022190 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:08.022196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:08.022239 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:08.022372 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:08.023101 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:08.024158 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:08.025522 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:08.026233 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:08.026391 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:08.029038 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:08.029043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:08.029043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:08.029054 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:08.031050 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:08.031505 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:08.031508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:08.031508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:08.031652 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:08.031864 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:08.031865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:08.031865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:08.032181 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:08.032357 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:08.032427 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 06:50:08.032579 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260806 06:50:08.033833 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:08.038582 27743 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:08.039103 27743 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:08.041000 27743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73f7216b04a54be48169db1f7638f261: Generated new TSK 0
I20260806 06:50:08.041185 27743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:08.044158 27733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:08.035267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59407 (local address 127.0.0.1:43177)
0806 06:50:08.035307 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:08.035309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:08.035479 (+ 170us) server_negotiation.cc:407] Connection header received
0806 06:50:08.035509 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:08.035510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:08.035520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:08.035584 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:08.036104 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:08.036738 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:08.037739 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:08.038043 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:08.038123 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:08.041513 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:08.041515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:08.041515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:08.041524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:08.043127 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:08.043434 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:08.043434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:08.043435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:08.043492 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:08.043664 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:08.043665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:08.043665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:08.043781 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:08.043872 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:08.043968 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 06:50:08.044070 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:50:08.045997 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27659
2026-08-06T06:50:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:08 dist-test-slave-5l3k krb5kdc[27761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:08 dist-test-slave-5l3k krb5kdc[27761](info): set up 2 sockets
Aug 06 06:50:08 dist-test-slave-5l3k krb5kdc[27761](info): commencing operation
krb5kdc: starting...
W20260806 06:50:09.096678 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:09 dist-test-slave-5l3k krb5kdc[27761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:09.215335 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:09.295127 27774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:09.295364 27774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:09.295403 27774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:09.296550 27774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:09.296618 27774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:09.296640 27774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:09.296661 27774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:09.296690 27774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:09.298169 27774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:09.298436 27774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:09.298669 27774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:09.303395 27782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:09.303395 27780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:09.303401 27779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:09.303622 27774 server_base.cc:1034] running on GCE node
I20260806 06:50:09.303835 27774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:09.304069 27774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:09.305207 27774 hybrid_clock.cc:648] HybridClock initialized: now 1785999009305184 us; error 38 us; skew 500 ppm
Aug 06 06:50:09 dist-test-slave-5l3k krb5kdc[27761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999009, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:09.307732 27774 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:09.308238 27774 webserver.cc:460] Webserver started at http://127.0.0.1:41781/ using document root <none> and password file <none>
I20260806 06:50:09.308459 27774 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:09.308504 27774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:09.308658 27774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:09.309662 27774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "f3cd3ce448d84062939ad7ced7105210"
format_stamp: "Formatted at 2026-08-06 06:50:09 on dist-test-slave-5l3k"
server_key: "138deba190aef1b2c5fda07325263c2b"
server_key_iv: "786a48def1cd01e459582a9788d93b22"
server_key_version: "encryptionkey@0"
I20260806 06:50:09.310036 27774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "f3cd3ce448d84062939ad7ced7105210"
format_stamp: "Formatted at 2026-08-06 06:50:09 on dist-test-slave-5l3k"
server_key: "138deba190aef1b2c5fda07325263c2b"
server_key_iv: "786a48def1cd01e459582a9788d93b22"
server_key_version: "encryptionkey@0"
I20260806 06:50:09.311410 27774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:09.312170 27789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:09.312357 27774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:09.312445 27774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "f3cd3ce448d84062939ad7ced7105210"
format_stamp: "Formatted at 2026-08-06 06:50:09 on dist-test-slave-5l3k"
server_key: "138deba190aef1b2c5fda07325263c2b"
server_key_iv: "786a48def1cd01e459582a9788d93b22"
server_key_version: "encryptionkey@0"
I20260806 06:50:09.312497 27774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:09.330206 27774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:09.332653 27774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:09.332818 27774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:09.336731 27774 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41309
I20260806 06:50:09.336781 27844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41309 every 8 connection(s)
I20260806 06:50:09.337132 27774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:09.338145 27845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:09.339993 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27774
I20260806 06:50:09.340118 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:09.340315 6373 external_mini_cluster.cc:1428] Setting key 39a7c18bba84db98efd78a590f0c1601
I20260806 06:50:09.340865 27845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: Bootstrap starting.
I20260806 06:50:09.341537 27845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:09.341837 27845 log.cc:824] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:09.342571 27845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: No bootstrap required, opened a new log
I20260806 06:50:09.344308 27845 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260806 06:50:09.344476 27845 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:09.344559 27845 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3cd3ce448d84062939ad7ced7105210, State: Initialized, Role: FOLLOWER
I20260806 06:50:09.344720 27845 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260806 06:50:09.344832 27845 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:09.344875 27845 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:09.344933 27845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:09.345651 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:09.340824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34737 (local address 127.0.0.1:41309)
0806 06:50:09.341003 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:09.341012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:09.341047 (+ 35us) server_negotiation.cc:407] Connection header received
0806 06:50:09.341478 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:09.341484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:09.341536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:09.341682 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:09.341858 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:09.342911 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:09.343742 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:09.344149 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:09.344268 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:09.344959 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:09.344962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:09.344963 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:09.345180 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:09.345215 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:09.345229 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:09.345299 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:09.345357 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:50:09.345420 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":230,"threads_started":1}
I20260806 06:50:09.345628 27845 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260806 06:50:09.345763 27845 leader_election.cc:302] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3cd3ce448d84062939ad7ced7105210; no voters:
I20260806 06:50:09.345922 27845 leader_election.cc:288] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:09.345976 27851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:09.346109 27851 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [term 1 LEADER]: Becoming Leader. State: Replica: f3cd3ce448d84062939ad7ced7105210, State: Running, Role: LEADER
I20260806 06:50:09.346148 27845 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:09.346192 27851 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260806 06:50:09.346576 27851 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3cd3ce448d84062939ad7ced7105210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } } }
I20260806 06:50:09.346642 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:09.346704 27851 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:09.346846 27851 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3cd3ce448d84062939ad7ced7105210. Latest consensus state: current_term: 1 leader_uuid: "f3cd3ce448d84062939ad7ced7105210" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cd3ce448d84062939ad7ced7105210" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } } }
I20260806 06:50:09.346895 27851 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:09.347976 27868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:09.348021 27868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:09.348086 27859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:09.348287 27859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:09.349735 27859 catalog_manager.cc:1347] Generated new cluster ID: aaed043598ff44d5a288bd34e5cf6242
I20260806 06:50:09.349786 27859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:09.353875 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:09.348260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48379 (local address 127.0.0.1:41309)
0806 06:50:09.348292 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:09.348294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:09.350657 (+ 2363us) server_negotiation.cc:407] Connection header received
0806 06:50:09.350708 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:09.350709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:09.350722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:09.350778 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:09.350948 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:09.351647 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:09.352669 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:09.352927 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:09.352998 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:09.353648 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:09.353649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:09.353649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:09.353653 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:09.353666 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:09.353670 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:09.353718 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:09.353784 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:50:09.353806 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:09.361145 27859 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:09.361817 27859 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:09.366261 27859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3cd3ce448d84062939ad7ced7105210: Generated new TSK 0
I20260806 06:50:09.366452 27859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:09.369629 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27774
2026-08-06T06:50:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:09 dist-test-slave-5l3k krb5kdc[27878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:09 dist-test-slave-5l3k krb5kdc[27878](info): set up 2 sockets
Aug 06 06:50:09 dist-test-slave-5l3k krb5kdc[27878](info): commencing operation
krb5kdc: starting...
W20260806 06:50:10.404029 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:10 dist-test-slave-5l3k krb5kdc[27878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:10.521373 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:10.600279 27892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:10.600497 27892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:10.600529 27892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:10.601588 27892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:10.601655 27892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:10.601675 27892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:10.601694 27892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:10.601722 27892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:10.603160 27892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:10.603435 27892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:10.603652 27892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:10.608433 27900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:10.608449 27897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:10.608506 27892 server_base.cc:1034] running on GCE node
W20260806 06:50:10.608458 27898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:10.608922 27892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:10.609210 27892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:10.610383 27892 hybrid_clock.cc:648] HybridClock initialized: now 1785999010610377 us; error 43 us; skew 500 ppm
Aug 06 06:50:10 dist-test-slave-5l3k krb5kdc[27878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999010, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:10.617317 27892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:10.617792 27892 webserver.cc:460] Webserver started at http://127.0.0.1:34289/ using document root <none> and password file <none>
I20260806 06:50:10.618018 27892 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:10.618113 27892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:10.618264 27892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:10.619313 27892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "cc504c8280a442638721601b06d2bb93"
format_stamp: "Formatted at 2026-08-06 06:50:10 on dist-test-slave-5l3k"
server_key: "b051f179712c8506a27dea550c41914f"
server_key_iv: "5457d296446598f6b08680b665847495"
server_key_version: "encryptionkey@0"
I20260806 06:50:10.619699 27892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "cc504c8280a442638721601b06d2bb93"
format_stamp: "Formatted at 2026-08-06 06:50:10 on dist-test-slave-5l3k"
server_key: "b051f179712c8506a27dea550c41914f"
server_key_iv: "5457d296446598f6b08680b665847495"
server_key_version: "encryptionkey@0"
I20260806 06:50:10.620926 27892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:50:10.621600 27907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:10.621776 27892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:10.621881 27892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "cc504c8280a442638721601b06d2bb93"
format_stamp: "Formatted at 2026-08-06 06:50:10 on dist-test-slave-5l3k"
server_key: "b051f179712c8506a27dea550c41914f"
server_key_iv: "5457d296446598f6b08680b665847495"
server_key_version: "encryptionkey@0"
I20260806 06:50:10.621950 27892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:10.625141 27892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:10.627276 27892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:10.627421 27892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:10.631366 27892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43295
I20260806 06:50:10.631395 27961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43295 every 8 connection(s)
I20260806 06:50:10.631699 27892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:10.632586 27962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:10.634686 27962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: Bootstrap starting.
I20260806 06:50:10.635215 27962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:10.635424 27962 log.cc:824] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:10.635990 27962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: No bootstrap required, opened a new log
I20260806 06:50:10.636186 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 27892
I20260806 06:50:10.636298 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:10.636497 6373 external_mini_cluster.cc:1428] Setting key 9a7bdb535b06af2c8857c07f266bbb65
I20260806 06:50:10.638283 27962 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260806 06:50:10.638504 27962 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:10.638582 27962 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc504c8280a442638721601b06d2bb93, State: Initialized, Role: FOLLOWER
I20260806 06:50:10.638741 27962 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260806 06:50:10.638839 27962 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:10.638873 27962 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:10.638948 27962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:10.639526 27962 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260806 06:50:10.639672 27962 leader_election.cc:302] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc504c8280a442638721601b06d2bb93; no voters:
I20260806 06:50:10.639818 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:10.636981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40283 (local address 127.0.0.1:43295)
0806 06:50:10.637166 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:10.637174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:10.637206 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:50:10.638473 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:10.638504 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:10.638539 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:10.638693 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:10.639361 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:10.639365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:10.639366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:10.639371 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:10.639416 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:10.639431 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:10.639512 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:10.639545 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 06:50:10.639600 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260806 06:50:10.639853 27962 leader_election.cc:288] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:10.639899 27968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:10.640091 27962 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:10.640089 27968 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [term 1 LEADER]: Becoming Leader. State: Replica: cc504c8280a442638721601b06d2bb93, State: Running, Role: LEADER
I20260806 06:50:10.640316 27968 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260806 06:50:10.640707 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:10.640648 27970 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc504c8280a442638721601b06d2bb93. Latest consensus state: current_term: 1 leader_uuid: "cc504c8280a442638721601b06d2bb93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } } }
I20260806 06:50:10.640753 27970 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:10.640645 27969 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc504c8280a442638721601b06d2bb93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc504c8280a442638721601b06d2bb93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } } }
I20260806 06:50:10.641892 27969 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:10.642105 27985 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:10.642159 27985 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:10.642247 27976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:10.642421 27976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:10.643920 27976 catalog_manager.cc:1347] Generated new cluster ID: 8521fe969b4b4aabb088fd2ebdd61075
I20260806 06:50:10.643971 27976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:10.644317 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:10.642286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57851 (local address 127.0.0.1:43295)
0806 06:50:10.642320 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:10.642322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:10.643260 (+ 938us) server_negotiation.cc:407] Connection header received
0806 06:50:10.643321 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:10.643322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:10.643326 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:10.643382 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:10.644055 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:10.644056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:10.644057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:10.644061 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:10.644085 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:10.644089 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:10.644119 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:10.644232 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 06:50:10.644244 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:50:10.668386 27976 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:10.668573 27976 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:10.672303 27976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc504c8280a442638721601b06d2bb93: Generated new TSK 0
I20260806 06:50:10.672492 27976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:10.675812 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 27892
2026-08-06T06:50:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:10 dist-test-slave-5l3k krb5kdc[27995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:10 dist-test-slave-5l3k krb5kdc[27995](info): set up 2 sockets
Aug 06 06:50:10 dist-test-slave-5l3k krb5kdc[27995](info): commencing operation
krb5kdc: starting...
W20260806 06:50:11.720362 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:11 dist-test-slave-5l3k krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:11.837400 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:11.917908 28009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:11.918150 28009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:11.918188 28009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:11.919343 28009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:11.919415 28009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:11.919438 28009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:11.919458 28009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:11.919488 28009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:11.920922 28009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:11.921195 28009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:11.921416 28009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:11.926149 28015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:11.926278 28014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:11.926183 28017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:11.926433 28009 server_base.cc:1034] running on GCE node
I20260806 06:50:11.926735 28009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:11.927006 28009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:11.928166 28009 hybrid_clock.cc:648] HybridClock initialized: now 1785999011928155 us; error 29 us; skew 500 ppm
Aug 06 06:50:11 dist-test-slave-5l3k krb5kdc[27995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:11.930703 28009 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:11.931169 28009 webserver.cc:460] Webserver started at http://127.0.0.1:37077/ using document root <none> and password file <none>
I20260806 06:50:11.931383 28009 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:11.931449 28009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:11.931612 28009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:11.932772 28009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "102b5f74094c457ab2b959ebf24796bd"
format_stamp: "Formatted at 2026-08-06 06:50:11 on dist-test-slave-5l3k"
server_key: "6ae08c9544b9090e24d939d001ef3862"
server_key_iv: "3fd06f9a84fa55f8b5cb4b4f0295e376"
server_key_version: "encryptionkey@0"
I20260806 06:50:11.933168 28009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "102b5f74094c457ab2b959ebf24796bd"
format_stamp: "Formatted at 2026-08-06 06:50:11 on dist-test-slave-5l3k"
server_key: "6ae08c9544b9090e24d939d001ef3862"
server_key_iv: "3fd06f9a84fa55f8b5cb4b4f0295e376"
server_key_version: "encryptionkey@0"
I20260806 06:50:11.934624 28009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 06:50:11.935425 28024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:11.935607 28009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:11.935715 28009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "102b5f74094c457ab2b959ebf24796bd"
format_stamp: "Formatted at 2026-08-06 06:50:11 on dist-test-slave-5l3k"
server_key: "6ae08c9544b9090e24d939d001ef3862"
server_key_iv: "3fd06f9a84fa55f8b5cb4b4f0295e376"
server_key_version: "encryptionkey@0"
I20260806 06:50:11.935788 28009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:11.955570 28009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:11.957331 28009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:11.957516 28009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:11.961781 28009 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37245
I20260806 06:50:11.961838 28078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37245 every 8 connection(s)
I20260806 06:50:11.962180 28009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:11.963191 28079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:11.965348 28079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd: Bootstrap starting.
I20260806 06:50:11.965919 28079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:11.966202 28079 log.cc:824] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:11.966970 28079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd: No bootstrap required, opened a new log
I20260806 06:50:11.968549 28079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260806 06:50:11.968698 28079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:11.968722 28079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 102b5f74094c457ab2b959ebf24796bd, State: Initialized, Role: FOLLOWER
I20260806 06:50:11.968873 28079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260806 06:50:11.968935 28079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:11.968956 28079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:11.969007 28079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:11.969527 28079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260806 06:50:11.969645 28079 leader_election.cc:302] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 102b5f74094c457ab2b959ebf24796bd; no voters:
I20260806 06:50:11.969834 28079 leader_election.cc:288] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:11.969882 28083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:11.970089 28079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:11.970211 28083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [term 1 LEADER]: Becoming Leader. State: Replica: 102b5f74094c457ab2b959ebf24796bd, State: Running, Role: LEADER
I20260806 06:50:11.970320 28083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260806 06:50:11.971021 28085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 102b5f74094c457ab2b959ebf24796bd. Latest consensus state: current_term: 1 leader_uuid: "102b5f74094c457ab2b959ebf24796bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260806 06:50:11.971130 28085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:11.971216 28084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "102b5f74094c457ab2b959ebf24796bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "102b5f74094c457ab2b959ebf24796bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260806 06:50:11.971284 28084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:11.971563 28094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:11.972141 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28009
I20260806 06:50:11.972209 28094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:11.972259 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:11.972472 6373 external_mini_cluster.cc:1428] Setting key 40caa6bf6e9323240ef313fa2bc51248
I20260806 06:50:11.974270 28094 catalog_manager.cc:1347] Generated new cluster ID: 4a2ef0f3c6ec4256a0e9ed5be08c5900
I20260806 06:50:11.974339 28094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:11 dist-test-slave-5l3k krb5kdc[27995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:50:11.984340 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:11.972929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:38243 (local address 127.0.0.1:37245)
0806 06:50:11.973097 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:11.973107 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:11.973145 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:50:11.973262 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:11.973269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:11.973325 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:11.973475 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:11.974272 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:11.975762 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:11.976865 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:11.977487 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:11.977578 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:11.980588 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:11.980592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:11.980593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:11.980608 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:11.982632 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:11.983060 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:11.983064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:11.983064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:11.983183 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:11.983408 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:11.983409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:11.983409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:11.983753 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:11.983916 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:11.984013 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:50:11.984118 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260806 06:50:11.985536 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:11.994594 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:11.987090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:49289 (local address 127.0.0.1:37245)
0806 06:50:11.987128 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:11.987130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:11.987276 (+ 146us) server_negotiation.cc:407] Connection header received
0806 06:50:11.987368 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:11.987369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:11.987378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:11.987405 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:11.988075 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:11.988868 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:11.989940 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:11.990246 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:11.990286 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:11.991733 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:11.991735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:11.991735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:11.991742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:11.993224 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:11.993661 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:11.993662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:11.993662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:11.993715 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:11.993944 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:11.993945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:11.993945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:11.994060 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:11.994145 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:11.994381 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 06:50:11.994444 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:12.002398 28094 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:12.002970 28094 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:12.014113 28094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 102b5f74094c457ab2b959ebf24796bd: Generated new TSK 0
I20260806 06:50:12.014333 28094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:12.020627 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28009
W20260806 06:50:12.027079 27993 connection.cc:535] client connection to 127.0.0.1:37245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37245): Connection reset by peer (error 104)
2026-08-06T06:50:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:12 dist-test-slave-5l3k krb5kdc[28112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:12 dist-test-slave-5l3k krb5kdc[28112](info): set up 2 sockets
Aug 06 06:50:12 dist-test-slave-5l3k krb5kdc[28112](info): commencing operation
krb5kdc: starting...
W20260806 06:50:13.060241 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:13 dist-test-slave-5l3k krb5kdc[28112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:13.180595 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:13.261658 28127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:13.261853 28127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:13.261876 28127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:13.263032 28127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:13.263072 28127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:13.263087 28127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:13.263129 28127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:13.263160 28127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:13.264595 28127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:13.264853 28127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:13.265069 28127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 06:50:13.269912 28127 server_base.cc:1034] running on GCE node
W20260806 06:50:13.269889 28133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:13.269899 28135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:13.269866 28132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:13.270394 28127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:13.270720 28127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:13.271864 28127 hybrid_clock.cc:648] HybridClock initialized: now 1785999013271853 us; error 28 us; skew 500 ppm
Aug 06 06:50:13 dist-test-slave-5l3k krb5kdc[28112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999013, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:13.274333 28127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:13.274784 28127 webserver.cc:460] Webserver started at http://127.0.0.1:43913/ using document root <none> and password file <none>
I20260806 06:50:13.275033 28127 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:13.275101 28127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:13.275251 28127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:13.276270 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "d0014fc413194de28ef6f26ef68bcf78"
format_stamp: "Formatted at 2026-08-06 06:50:13 on dist-test-slave-5l3k"
server_key: "df12b03f81b580130a88c242d4bfd85a"
server_key_iv: "7ae67f2b89468209709ec0aee6f483cf"
server_key_version: "encryptionkey@0"
I20260806 06:50:13.276625 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "d0014fc413194de28ef6f26ef68bcf78"
format_stamp: "Formatted at 2026-08-06 06:50:13 on dist-test-slave-5l3k"
server_key: "df12b03f81b580130a88c242d4bfd85a"
server_key_iv: "7ae67f2b89468209709ec0aee6f483cf"
server_key_version: "encryptionkey@0"
I20260806 06:50:13.277910 28127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:13.278671 28142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:13.278846 28127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:13.278966 28127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "d0014fc413194de28ef6f26ef68bcf78"
format_stamp: "Formatted at 2026-08-06 06:50:13 on dist-test-slave-5l3k"
server_key: "df12b03f81b580130a88c242d4bfd85a"
server_key_iv: "7ae67f2b89468209709ec0aee6f483cf"
server_key_version: "encryptionkey@0"
I20260806 06:50:13.279037 28127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:13.295492 28127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:13.297829 28127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:13.297994 28127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:13.302119 28127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39747
I20260806 06:50:13.302153 28197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39747 every 8 connection(s)
I20260806 06:50:13.302552 28127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:13.303697 28198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:13.305037 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28127
I20260806 06:50:13.305169 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:13.305387 6373 external_mini_cluster.cc:1428] Setting key f5389a15ab9faa3920a2e868fe95f270
I20260806 06:50:13.306996 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78: Bootstrap starting.
I20260806 06:50:13.307823 28198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:13.308172 28198 log.cc:824] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:13.309002 28198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78: No bootstrap required, opened a new log
I20260806 06:50:13.311488 28198 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260806 06:50:13.311687 28198 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:13.311749 28198 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0014fc413194de28ef6f26ef68bcf78, State: Initialized, Role: FOLLOWER
I20260806 06:50:13.311937 28198 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260806 06:50:13.312052 28198 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:13.312091 28198 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:13.312136 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:13.305955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:33865 (local address 127.0.0.1:39747)
0806 06:50:13.306204 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:13.306214 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:13.306251 (+ 37us) server_negotiation.cc:407] Connection header received
0806 06:50:13.306878 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:13.306887 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:13.306955 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:13.307134 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:13.307347 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:13.308857 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:13.309712 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:13.310431 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:13.310586 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:13.311230 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:13.311234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:13.311235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:13.311448 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:13.311487 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:13.311504 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:13.311601 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:13.311693 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 06:50:13.311866 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":127,"threads_started":1}
I20260806 06:50:13.312146 28198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:13.312880 28198 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260806 06:50:13.312999 28198 leader_election.cc:302] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0014fc413194de28ef6f26ef68bcf78; no voters:
I20260806 06:50:13.313180 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:13.313184 28198 leader_election.cc:288] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:13.313217 28204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:13.313400 28204 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [term 1 LEADER]: Becoming Leader. State: Replica: d0014fc413194de28ef6f26ef68bcf78, State: Running, Role: LEADER
I20260806 06:50:13.313495 28198 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:13.313589 28204 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260806 06:50:13.314016 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0014fc413194de28ef6f26ef68bcf78. Latest consensus state: current_term: 1 leader_uuid: "d0014fc413194de28ef6f26ef68bcf78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260806 06:50:13.314127 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:13.314021 28205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0014fc413194de28ef6f26ef68bcf78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0014fc413194de28ef6f26ef68bcf78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260806 06:50:13.314297 28205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:13.315176 28213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:13.315783 28213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:13.317409 28213 catalog_manager.cc:1347] Generated new cluster ID: fc03342752344ba6aede716e441e78e6
I20260806 06:50:13.317459 28213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:13.318781 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:13.315024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:34165 (local address 127.0.0.1:39747)
0806 06:50:13.315055 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:13.315057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:13.315065 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:13.315678 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:13.315679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:13.315693 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:13.315754 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:13.315883 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:13.316365 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:13.317265 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:13.317632 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:13.317716 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:13.318460 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:13.318462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:13.318462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:13.318467 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:13.318511 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:13.318517 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:13.318597 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:13.318670 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 06:50:13.318681 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:50:13.346170 28213 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:13.346807 28213 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:13.355875 28213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0014fc413194de28ef6f26ef68bcf78: Generated new TSK 0
I20260806 06:50:13.356055 28213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:13.362061 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28127
W20260806 06:50:13.368113 28110 connection.cc:535] client connection to 127.0.0.1:39747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39747): Connection reset by peer (error 104)
2026-08-06T06:50:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:13 dist-test-slave-5l3k krb5kdc[28230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:13 dist-test-slave-5l3k krb5kdc[28230](info): set up 2 sockets
Aug 06 06:50:13 dist-test-slave-5l3k krb5kdc[28230](info): commencing operation
krb5kdc: starting...
W20260806 06:50:14.460855 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:14 dist-test-slave-5l3k krb5kdc[28230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:14.580590 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:14.660562 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:14.660795 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:14.660835 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:14.661949 28244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:14.662040 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:14.662068 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:14.662101 28244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:14.662127 28244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:14.663615 28244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:14.663888 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:14.664120 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:14.668941 28249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:14.668946 28252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:14.668928 28250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:14.669123 28244 server_base.cc:1034] running on GCE node
I20260806 06:50:14.669440 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:14.669746 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:14.670920 28244 hybrid_clock.cc:648] HybridClock initialized: now 1785999014670905 us; error 45 us; skew 500 ppm
Aug 06 06:50:14 dist-test-slave-5l3k krb5kdc[28230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999014, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:14.673333 28244 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:14.673753 28244 webserver.cc:460] Webserver started at http://127.0.0.1:33827/ using document root <none> and password file <none>
I20260806 06:50:14.674007 28244 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:14.674049 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:14.674191 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:14.675206 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "3e8f059658a14617bf2924ecec3ddcc7"
format_stamp: "Formatted at 2026-08-06 06:50:14 on dist-test-slave-5l3k"
server_key: "528843806f0f8e3c4725535694811c4a"
server_key_iv: "2d1f79a5045957669564230038ac9622"
server_key_version: "encryptionkey@0"
I20260806 06:50:14.675527 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "3e8f059658a14617bf2924ecec3ddcc7"
format_stamp: "Formatted at 2026-08-06 06:50:14 on dist-test-slave-5l3k"
server_key: "528843806f0f8e3c4725535694811c4a"
server_key_iv: "2d1f79a5045957669564230038ac9622"
server_key_version: "encryptionkey@0"
I20260806 06:50:14.676939 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:50:14.677709 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:14.677930 28244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:14.678036 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "3e8f059658a14617bf2924ecec3ddcc7"
format_stamp: "Formatted at 2026-08-06 06:50:14 on dist-test-slave-5l3k"
server_key: "528843806f0f8e3c4725535694811c4a"
server_key_iv: "2d1f79a5045957669564230038ac9622"
server_key_version: "encryptionkey@0"
I20260806 06:50:14.678117 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:14.681376 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:14.683223 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:14.683374 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:14.687419 28244 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34251
I20260806 06:50:14.687450 28315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34251 every 8 connection(s)
I20260806 06:50:14.687803 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:14.688793 28316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:14.690950 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7: Bootstrap starting.
I20260806 06:50:14.691542 28316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:14.691817 28316 log.cc:824] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:14.692528 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7: No bootstrap required, opened a new log
I20260806 06:50:14.694144 28316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 06:50:14.694272 28316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:14.694295 28316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e8f059658a14617bf2924ecec3ddcc7, State: Initialized, Role: FOLLOWER
I20260806 06:50:14.694461 28316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 06:50:14.694591 28316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:14.694638 28316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:14.694694 28316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:14.694983 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28244
I20260806 06:50:14.695139 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:14.695314 28316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 06:50:14.695340 6373 external_mini_cluster.cc:1428] Setting key 78a269aa4525a4166d0f797cbeab3660
I20260806 06:50:14.695426 28316 leader_election.cc:302] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e8f059658a14617bf2924ecec3ddcc7; no voters:
I20260806 06:50:14.695600 28316 leader_election.cc:288] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:14.695636 28319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:14.695839 28319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [term 1 LEADER]: Becoming Leader. State: Replica: 3e8f059658a14617bf2924ecec3ddcc7, State: Running, Role: LEADER
I20260806 06:50:14.695880 28316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:14.695995 28319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260806 06:50:14.696358 28320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e8f059658a14617bf2924ecec3ddcc7. Latest consensus state: current_term: 1 leader_uuid: "3e8f059658a14617bf2924ecec3ddcc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260806 06:50:14.696506 28320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:14.697003 28322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e8f059658a14617bf2924ecec3ddcc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e8f059658a14617bf2924ecec3ddcc7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260806 06:50:14.697124 28322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:14.697201 28325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:14.697703 28325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:14.697932 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:14.695828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:51497 (local address 127.0.0.1:34251)
0806 06:50:14.696144 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:14.696152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:14.696184 (+ 32us) server_negotiation.cc:407] Connection header received
0806 06:50:14.696297 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:14.696304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:14.696334 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:14.696493 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:14.697242 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:14.697245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:14.697246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:14.697251 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:14.697313 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:14.697331 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:14.697411 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:14.697501 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:50:14.697637 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":70,"threads_started":1}
I20260806 06:50:14.698941 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:14.699685 28325 catalog_manager.cc:1347] Generated new cluster ID: e442fc827fb4424ea9085c77cb39d0a0
I20260806 06:50:14.699765 28325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:14.701622 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:14.700514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:40891 (local address 127.0.0.1:34251)
0806 06:50:14.700549 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:14.700551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:14.700726 (+ 175us) server_negotiation.cc:407] Connection header received
0806 06:50:14.700757 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:14.700758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:14.700763 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:14.700827 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:14.701385 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:14.701386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:14.701386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:14.701391 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:14.701405 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:14.701410 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:14.701473 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:14.701531 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:50:14.701544 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:14.721915 28325 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:14.722152 28325 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:14.730855 28325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e8f059658a14617bf2924ecec3ddcc7: Generated new TSK 0
I20260806 06:50:14.731052 28325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:14.735106 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28244
2026-08-06T06:50:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:14 dist-test-slave-5l3k krb5kdc[28349](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:14 dist-test-slave-5l3k krb5kdc[28349](info): set up 2 sockets
Aug 06 06:50:14 dist-test-slave-5l3k krb5kdc[28349](info): commencing operation
krb5kdc: starting...
W20260806 06:50:15.770553 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:15 dist-test-slave-5l3k krb5kdc[28349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:15.889490 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:15.971431 28363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:15.971669 28363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:15.971711 28363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:15.972925 28363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:15.972991 28363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:15.973016 28363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:15.973037 28363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:15.973067 28363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:15.974527 28363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:15.974802 28363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:15.975029 28363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:15.979974 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:15.979952 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:15.980014 28370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:15.980196 28363 server_base.cc:1034] running on GCE node
I20260806 06:50:15.980409 28363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:15.980648 28363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:15.981791 28363 hybrid_clock.cc:648] HybridClock initialized: now 1785999015981777 us; error 27 us; skew 500 ppm
Aug 06 06:50:15 dist-test-slave-5l3k krb5kdc[28349](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:15.987799 28363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:15.988252 28363 webserver.cc:460] Webserver started at http://127.0.0.1:36599/ using document root <none> and password file <none>
I20260806 06:50:15.988433 28363 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:15.988473 28363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:15.988560 28363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:15.989569 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "cb808699c312476f989104d8d0639f13"
format_stamp: "Formatted at 2026-08-06 06:50:15 on dist-test-slave-5l3k"
server_key: "f4159bdbc553fec2e72b1bc814f400cd"
server_key_iv: "651fbeaa6406565191d44d8e257558ff"
server_key_version: "encryptionkey@0"
I20260806 06:50:15.989859 28363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "cb808699c312476f989104d8d0639f13"
format_stamp: "Formatted at 2026-08-06 06:50:15 on dist-test-slave-5l3k"
server_key: "f4159bdbc553fec2e72b1bc814f400cd"
server_key_iv: "651fbeaa6406565191d44d8e257558ff"
server_key_version: "encryptionkey@0"
I20260806 06:50:15.991191 28363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:15.991871 28380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:15.992061 28363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:15.992161 28363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "cb808699c312476f989104d8d0639f13"
format_stamp: "Formatted at 2026-08-06 06:50:15 on dist-test-slave-5l3k"
server_key: "f4159bdbc553fec2e72b1bc814f400cd"
server_key_iv: "651fbeaa6406565191d44d8e257558ff"
server_key_version: "encryptionkey@0"
I20260806 06:50:15.992218 28363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:16.003139 28363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:16.004892 28363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:16.004993 28363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:16.008944 28363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39129
I20260806 06:50:16.008998 28436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39129 every 8 connection(s)
I20260806 06:50:16.009315 28363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:16.010320 28437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:16.012527 28437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13: Bootstrap starting.
I20260806 06:50:16.013108 28437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:16.013392 28437 log.cc:824] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:16.014102 28437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13: No bootstrap required, opened a new log
I20260806 06:50:16.014364 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28363
I20260806 06:50:16.014474 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:16.014675 6373 external_mini_cluster.cc:1428] Setting key de3fb1f1ef79d4e8cd0131e23ede2ae7
I20260806 06:50:16.016412 28437 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260806 06:50:16.016621 28437 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:16.016659 28437 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb808699c312476f989104d8d0639f13, State: Initialized, Role: FOLLOWER
I20260806 06:50:16.016783 28437 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260806 06:50:16.016872 28437 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:16.016904 28437 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:16.016990 28437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:16.017562 28437 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260806 06:50:16.017670 28437 leader_election.cc:302] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb808699c312476f989104d8d0639f13; no voters:
I20260806 06:50:16.017851 28437 leader_election.cc:288] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:16.017921 28443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:16.018100 28437 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:16.018100 28443 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [term 1 LEADER]: Becoming Leader. State: Replica: cb808699c312476f989104d8d0639f13, State: Running, Role: LEADER
I20260806 06:50:16.018324 28443 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } }
I20260806 06:50:16.018756 28444 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb808699c312476f989104d8d0639f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } } }
I20260806 06:50:16.018893 28444 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:16.019064 28444 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb808699c312476f989104d8d0639f13. Latest consensus state: current_term: 1 leader_uuid: "cb808699c312476f989104d8d0639f13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb808699c312476f989104d8d0639f13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39129 } } }
I20260806 06:50:16.019120 28453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:16.019181 28444 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:16.019731 28453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:16.021821 28453 catalog_manager.cc:1347] Generated new cluster ID: 2a9c756f5c944f33bea2b3c41c6379d4
I20260806 06:50:16.021874 28453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:16 dist-test-slave-5l3k krb5kdc[28349](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 06:50:16.027051 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:16.015154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:58597 (local address 127.0.0.1:39129)
0806 06:50:16.015345 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:16.015354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:16.015388 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:50:16.016621 (+ 1233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:16.016631 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:16.016697 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:16.016840 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:16.017569 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:16.019042 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:16.020256 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:16.020838 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:16.020945 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:16.023402 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:16.023406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:16.023406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:16.023417 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:16.025463 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:16.025880 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:16.025885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:16.025890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:16.025975 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:16.026211 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:16.026212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:16.026213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:16.026434 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:16.026684 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:16.026781 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:50:16.026887 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260806 06:50:16.028084 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:16.029902 28453 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:16.030588 28453 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:16.035499 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:16.029435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:60433 (local address 127.0.0.1:39129)
0806 06:50:16.029470 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:16.029472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:16.029588 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:50:16.029690 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:16.029691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:16.029701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:16.029748 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:16.030302 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:16.030935 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:16.031630 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:16.031847 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:16.031915 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:16.032944 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:16.032945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:16.032946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:16.032953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:16.034302 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:16.034721 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:16.034722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:16.034722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:16.034777 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:16.035003 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:16.035004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:16.035005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:16.035117 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:16.035220 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:16.035332 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 06:50:16.035390 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":18}
I20260806 06:50:16.037350 28453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb808699c312476f989104d8d0639f13: Generated new TSK 0
I20260806 06:50:16.037534 28453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:16.040407 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28363
W20260806 06:50:16.046581 28347 connection.cc:535] client connection to 127.0.0.1:39129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39129): Connection reset by peer (error 104)
2026-08-06T06:50:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:16 dist-test-slave-5l3k krb5kdc[28472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:16 dist-test-slave-5l3k krb5kdc[28472](info): set up 2 sockets
Aug 06 06:50:16 dist-test-slave-5l3k krb5kdc[28472](info): commencing operation
krb5kdc: starting...
W20260806 06:50:17.084178 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:17 dist-test-slave-5l3k krb5kdc[28472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:17.201622 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:17.281010 28485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:17.281201 28485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:17.281225 28485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:17.282258 28485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:17.282291 28485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:17.282305 28485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:17.282317 28485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:17.282330 28485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:17.283751 28485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40043
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:17.283946 28485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:17.284103 28485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:17.288672 28493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:17.288681 28490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:17.288681 28491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:17.288810 28485 server_base.cc:1034] running on GCE node
I20260806 06:50:17.289108 28485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:17.289392 28485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:17.290544 28485 hybrid_clock.cc:648] HybridClock initialized: now 1785999017290533 us; error 36 us; skew 500 ppm
Aug 06 06:50:17 dist-test-slave-5l3k krb5kdc[28472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999017, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:17.292832 28485 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:17.293249 28485 webserver.cc:460] Webserver started at http://127.0.0.1:34471/ using document root <none> and password file <none>
I20260806 06:50:17.293468 28485 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:17.293511 28485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:17.293707 28485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:17.294785 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "d150a68006444293be95c652dce5fdea"
format_stamp: "Formatted at 2026-08-06 06:50:17 on dist-test-slave-5l3k"
server_key: "5b36d949548065911acb2f70f80eed1d"
server_key_iv: "061717901ed15a0fdd98d466288490e9"
server_key_version: "encryptionkey@0"
I20260806 06:50:17.295126 28485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "d150a68006444293be95c652dce5fdea"
format_stamp: "Formatted at 2026-08-06 06:50:17 on dist-test-slave-5l3k"
server_key: "5b36d949548065911acb2f70f80eed1d"
server_key_iv: "061717901ed15a0fdd98d466288490e9"
server_key_version: "encryptionkey@0"
I20260806 06:50:17.296482 28485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:17.297155 28500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:17.297300 28485 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:17.297399 28485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "d150a68006444293be95c652dce5fdea"
format_stamp: "Formatted at 2026-08-06 06:50:17 on dist-test-slave-5l3k"
server_key: "5b36d949548065911acb2f70f80eed1d"
server_key_iv: "061717901ed15a0fdd98d466288490e9"
server_key_version: "encryptionkey@0"
I20260806 06:50:17.297468 28485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:17.311053 28485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:17.313033 28485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:17.313181 28485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:17.317158 28485 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40043
I20260806 06:50:17.317219 28556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40043 every 8 connection(s)
I20260806 06:50:17.317544 28485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:17.318635 28557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:17.321067 28557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea: Bootstrap starting.
I20260806 06:50:17.321621 28557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:17.321882 28557 log.cc:824] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:17.322651 28557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea: No bootstrap required, opened a new log
I20260806 06:50:17.324337 28557 raft_consensus.cc:348] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } }
I20260806 06:50:17.324477 28557 raft_consensus.cc:374] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:17.324501 28557 raft_consensus.cc:729] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d150a68006444293be95c652dce5fdea, State: Initialized, Role: FOLLOWER
I20260806 06:50:17.324649 28557 consensus_queue.cc:260] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } }
I20260806 06:50:17.324741 28557 raft_consensus.cc:388] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:17.324771 28557 raft_consensus.cc:482] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:17.324827 28557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:17.325382 28557 raft_consensus.cc:504] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } }
I20260806 06:50:17.325472 28557 leader_election.cc:302] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d150a68006444293be95c652dce5fdea; no voters:
I20260806 06:50:17.325613 28557 leader_election.cc:288] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:17.325662 28560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:17.325845 28560 raft_consensus.cc:686] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [term 1 LEADER]: Becoming Leader. State: Replica: d150a68006444293be95c652dce5fdea, State: Running, Role: LEADER
I20260806 06:50:17.325970 28560 consensus_queue.cc:237] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } }
I20260806 06:50:17.326097 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28485
I20260806 06:50:17.326226 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:17.326352 28561 sys_catalog.cc:455] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d150a68006444293be95c652dce5fdea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } } }
I20260806 06:50:17.326459 6373 external_mini_cluster.cc:1428] Setting key 711cf3637eaa4fbb30e1055ad224c737
I20260806 06:50:17.326514 28563 sys_catalog.cc:455] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [sys.catalog]: SysCatalogTable state changed. Reason: New leader d150a68006444293be95c652dce5fdea. Latest consensus state: current_term: 1 leader_uuid: "d150a68006444293be95c652dce5fdea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d150a68006444293be95c652dce5fdea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40043 } } }
I20260806 06:50:17.326526 28561 sys_catalog.cc:458] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:17.326583 28563 sys_catalog.cc:458] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:17.326938 28557 sys_catalog.cc:564] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:17.327039 28565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:17.327617 28565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:17.329375 28565 catalog_manager.cc:1347] Generated new cluster ID: 47b678b3b07a4d649fb993357c1c4195
I20260806 06:50:17.329435 28565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:17.332487 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:17.327178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:38155 (local address 127.0.0.1:40043)
0806 06:50:17.327372 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:17.327382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:17.327422 (+ 40us) server_negotiation.cc:407] Connection header received
0806 06:50:17.327532 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:17.327541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:17.327595 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:17.327990 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:17.328092 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:17.329677 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:17.330516 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:17.330944 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:17.331076 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:17.331814 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:17.331818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:17.331819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:17.332007 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:17.332035 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:17.332046 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:17.332146 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:17.332236 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:50:17.332263 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 06:50:17.333340 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:17.337448 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:17.334712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:35583 (local address 127.0.0.1:40043)
0806 06:50:17.334831 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:17.334833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:17.334862 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:50:17.334929 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:17.334930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:17.334939 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:17.334977 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:17.335102 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:17.335587 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:17.336295 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:17.336517 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:17.336591 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:17.337235 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:17.337236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:17.337236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:17.337241 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:17.337253 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:17.337257 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:17.337308 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:17.337366 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 06:50:17.337378 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 06:50:17.338249 28565 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:17.339013 28565 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:17.358553 28565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d150a68006444293be95c652dce5fdea: Generated new TSK 0
I20260806 06:50:17.358754 28565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:17.363734 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28485
W20260806 06:50:17.370431 28470 connection.cc:535] client connection to 127.0.0.1:40043 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40043): Connection reset by peer (error 104)
2026-08-06T06:50:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:17 dist-test-slave-5l3k krb5kdc[28589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:17 dist-test-slave-5l3k krb5kdc[28589](info): set up 2 sockets
Aug 06 06:50:17 dist-test-slave-5l3k krb5kdc[28589](info): commencing operation
krb5kdc: starting...
W20260806 06:50:18.400031 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:18 dist-test-slave-5l3k krb5kdc[28589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 06:50:18.518842 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:18.600903 28603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:18.601089 28603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:18.601109 28603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:18.602226 28603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:18.602267 28603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:18.602280 28603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:18.602316 28603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:18.602345 28603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:18.603822 28603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:18.604072 28603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:18.604290 28603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:18.609026 28612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:18.609113 28609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:18.609154 28603 server_base.cc:1034] running on GCE node
W20260806 06:50:18.609047 28610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:18.609536 28603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:18.609813 28603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:18.610952 28603 hybrid_clock.cc:648] HybridClock initialized: now 1785999018610921 us; error 50 us; skew 500 ppm
Aug 06 06:50:18 dist-test-slave-5l3k krb5kdc[28589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:18.613299 28603 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 06:50:18.613782 28603 webserver.cc:460] Webserver started at http://127.0.0.1:44451/ using document root <none> and password file <none>
I20260806 06:50:18.613950 28603 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:18.613988 28603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:18.614092 28603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:18.615113 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "719ec9b736004499aec73459a612793c"
format_stamp: "Formatted at 2026-08-06 06:50:18 on dist-test-slave-5l3k"
server_key: "f8eccfa9d5b6ee5318b0c6cb22880b3a"
server_key_iv: "22e32517c86a2fae168db812297d152a"
server_key_version: "encryptionkey@0"
I20260806 06:50:18.615383 28603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "719ec9b736004499aec73459a612793c"
format_stamp: "Formatted at 2026-08-06 06:50:18 on dist-test-slave-5l3k"
server_key: "f8eccfa9d5b6ee5318b0c6cb22880b3a"
server_key_iv: "22e32517c86a2fae168db812297d152a"
server_key_version: "encryptionkey@0"
I20260806 06:50:18.616755 28603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:18.617467 28619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:18.617677 28603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:18.617789 28603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "719ec9b736004499aec73459a612793c"
format_stamp: "Formatted at 2026-08-06 06:50:18 on dist-test-slave-5l3k"
server_key: "f8eccfa9d5b6ee5318b0c6cb22880b3a"
server_key_iv: "22e32517c86a2fae168db812297d152a"
server_key_version: "encryptionkey@0"
I20260806 06:50:18.617842 28603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:18.620926 28603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:18.622722 28603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:18.622841 28603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:18.626679 28603 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39057
I20260806 06:50:18.626797 28673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39057 every 8 connection(s)
I20260806 06:50:18.627066 28603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:18.628083 28674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:18.630164 28674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: Bootstrap starting.
I20260806 06:50:18.630761 28674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:18.630999 28674 log.cc:824] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:18.631619 28674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: No bootstrap required, opened a new log
I20260806 06:50:18.633194 28674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260806 06:50:18.633320 28674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:18.633342 28674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 719ec9b736004499aec73459a612793c, State: Initialized, Role: FOLLOWER
I20260806 06:50:18.633415 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28603
I20260806 06:50:18.633497 28674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260806 06:50:18.633543 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:18.633553 28674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:18.633612 28674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:18.633665 28674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:18.633750 6373 external_mini_cluster.cc:1428] Setting key d2c6e583ff9cc479329aece108a22110
I20260806 06:50:18.634338 28674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260806 06:50:18.634455 28674 leader_election.cc:302] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 719ec9b736004499aec73459a612793c; no voters:
I20260806 06:50:18.634639 28674 leader_election.cc:288] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:18.634827 28679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:18.634896 28674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:18.635322 28679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [term 1 LEADER]: Becoming Leader. State: Replica: 719ec9b736004499aec73459a612793c, State: Running, Role: LEADER
I20260806 06:50:18.635497 28679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } }
I20260806 06:50:18.635998 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:18.634234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:56321 (local address 127.0.0.1:39057)
0806 06:50:18.634428 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:18.634437 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:18.634468 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:50:18.634599 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:18.634604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:18.634632 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:18.634809 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:18.635442 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:18.635445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:18.635446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:18.635450 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:18.635497 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:18.635510 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:18.635578 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:18.635738 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:50:18.635784 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260806 06:50:18.635989 28679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 719ec9b736004499aec73459a612793c. Latest consensus state: current_term: 1 leader_uuid: "719ec9b736004499aec73459a612793c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } } }
I20260806 06:50:18.636082 28679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:18.636264 28681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "719ec9b736004499aec73459a612793c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "719ec9b736004499aec73459a612793c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39057 } } }
I20260806 06:50:18.636718 28681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:18.637022 28694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:18.637104 28694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:18.637111 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:18.637169 28695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:18.637306 28695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:18.638945 28695 catalog_manager.cc:1347] Generated new cluster ID: 74b8a5f7190a42b0bc3a5ad5b317eee3
I20260806 06:50:18.639017 28695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:18.639956 28678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:18.638583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.235:46195 (local address 127.0.0.1:39057)
0806 06:50:18.638620 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:18.638622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:18.639052 (+ 430us) server_negotiation.cc:407] Connection header received
0806 06:50:18.639160 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:18.639161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:18.639166 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:18.639218 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:18.639764 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:18.639765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:18.639765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:18.639769 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:18.639782 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:18.639785 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:18.639834 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:18.639893 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 06:50:18.639904 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:18.644591 28695 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:18.644754 28695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:18.648663 28695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 719ec9b736004499aec73459a612793c: Generated new TSK 0
I20260806 06:50:18.648833 28695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:18.654325 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28603
2026-08-06T06:50:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:18 dist-test-slave-5l3k krb5kdc[28706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:18 dist-test-slave-5l3k krb5kdc[28706](info): set up 2 sockets
Aug 06 06:50:18 dist-test-slave-5l3k krb5kdc[28706](info): commencing operation
krb5kdc: starting...
W20260806 06:50:19.700814 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:19 dist-test-slave-5l3k krb5kdc[28706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:19.821913 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42665
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:42381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:19.902616 28721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:19.902848 28721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:19.902886 28721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:19.904036 28721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:19.904117 28721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:19.904146 28721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:19.904179 28721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:19.904204 28721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:19.905766 28721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:42381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42665
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:19.906070 28721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:19.906301 28721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:19.911171 28727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:19.911176 28726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:19.911446 28721 server_base.cc:1034] running on GCE node
W20260806 06:50:19.911195 28729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:19.911728 28721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:19.912048 28721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:19.913187 28721 hybrid_clock.cc:648] HybridClock initialized: now 1785999019913163 us; error 36 us; skew 500 ppm
Aug 06 06:50:19 dist-test-slave-5l3k krb5kdc[28706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:19.915702 28721 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:19.916144 28721 webserver.cc:460] Webserver started at http://127.6.57.126:38981/ using document root <none> and password file <none>
I20260806 06:50:19.916375 28721 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:19.916441 28721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:19.916586 28721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:19.917583 28721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "6e1346090cb14189a4103cc9b3a97edb"
format_stamp: "Formatted at 2026-08-06 06:50:19 on dist-test-slave-5l3k"
server_key: "6fe7e6d867d649a80c6edbb9c39160af"
server_key_iv: "8e214d4254314142ab62a279c222ba97"
server_key_version: "encryptionkey@0"
I20260806 06:50:19.917989 28721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "6e1346090cb14189a4103cc9b3a97edb"
format_stamp: "Formatted at 2026-08-06 06:50:19 on dist-test-slave-5l3k"
server_key: "6fe7e6d867d649a80c6edbb9c39160af"
server_key_iv: "8e214d4254314142ab62a279c222ba97"
server_key_version: "encryptionkey@0"
I20260806 06:50:19.919375 28721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 06:50:19.920140 28736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:19.920306 28721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:19.920413 28721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "6e1346090cb14189a4103cc9b3a97edb"
format_stamp: "Formatted at 2026-08-06 06:50:19 on dist-test-slave-5l3k"
server_key: "6fe7e6d867d649a80c6edbb9c39160af"
server_key_iv: "8e214d4254314142ab62a279c222ba97"
server_key_version: "encryptionkey@0"
I20260806 06:50:19.920485 28721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:19.936895 28721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:19.938757 28721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:19.938913 28721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:19.942696 28721 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:42665
I20260806 06:50:19.942762 28790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:42665 every 8 connection(s)
I20260806 06:50:19.943104 28721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:19.943962 28791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:19.946267 28791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: Bootstrap starting.
I20260806 06:50:19.946913 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28721
I20260806 06:50:19.946960 28791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:19.947054 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:19.947302 6373 external_mini_cluster.cc:1428] Setting key 45cdccf24dfc63822644f193e9bb4a85
I20260806 06:50:19.947335 28791 log.cc:824] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:19.948117 28791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: No bootstrap required, opened a new log
I20260806 06:50:19.950088 28791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } }
I20260806 06:50:19.950227 28791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:19.950249 28791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1346090cb14189a4103cc9b3a97edb, State: Initialized, Role: FOLLOWER
I20260806 06:50:19.950394 28791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } }
I20260806 06:50:19.950474 28791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:19.950526 28791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:19.950582 28791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:19.951078 28791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } }
I20260806 06:50:19.951210 28791 leader_election.cc:302] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e1346090cb14189a4103cc9b3a97edb; no voters:
I20260806 06:50:19.951395 28791 leader_election.cc:288] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:19.951414 28796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:19.951555 28796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [term 1 LEADER]: Becoming Leader. State: Replica: 6e1346090cb14189a4103cc9b3a97edb, State: Running, Role: LEADER
I20260806 06:50:19.951690 28791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:19.951717 28796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } }
I20260806 06:50:19.952104 28796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e1346090cb14189a4103cc9b3a97edb. Latest consensus state: current_term: 1 leader_uuid: "6e1346090cb14189a4103cc9b3a97edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } } }
I20260806 06:50:19.952131 28797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e1346090cb14189a4103cc9b3a97edb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e1346090cb14189a4103cc9b3a97edb" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42665 } } }
I20260806 06:50:19.952242 28796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:19.952260 28797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:19.953631 28813 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:19.953739 28813 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:19.953806 28804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:19.953969 28804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 06:50:19 dist-test-slave-5l3k krb5kdc[28706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:19.955673 28804 catalog_manager.cc:1347] Generated new cluster ID: 0d19a86350d34b8db00df5cd1ded4ba4
I20260806 06:50:19.955729 28804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:19.958518 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:19.947759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40753 (local address 127.6.57.126:42665)
0806 06:50:19.947955 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:19.947966 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:19.947997 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:50:19.948482 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:19.948489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:19.948535 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:19.948680 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:19.949312 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:19.950197 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:19.951093 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:19.951753 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:19.953628 (+ 1875us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:19.954675 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:19.954679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:19.954679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:19.954690 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:19.956602 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:19.956988 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:19.956991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:19.956991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:19.957108 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:19.957425 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:19.957426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:19.957426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:19.957703 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:19.958123 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:19.958206 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 06:50:19.958303 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260806 06:50:19.964905 28804 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:19.965632 28804 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:19.968611 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:19.973352 28804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e1346090cb14189a4103cc9b3a97edb: Generated new TSK 0
I20260806 06:50:19.973543 28804 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:19.977635 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:19.970782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39865 (local address 127.6.57.126:42665)
0806 06:50:19.970820 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:19.970822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:19.970996 (+ 174us) server_negotiation.cc:407] Connection header received
0806 06:50:19.971059 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:19.971061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:19.971070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:19.971118 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:19.971642 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:19.972262 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:19.973328 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:19.973607 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:19.973668 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:19.975003 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:19.975005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:19.975005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:19.975021 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:19.976488 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:19.976854 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:19.976855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:19.976856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:19.976916 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:19.977140 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:19.977142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:19.977142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:19.977249 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:19.977361 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:19.977471 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 06:50:19.977545 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:19.979590 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28721
W20260806 06:50:19.985265 28704 connection.cc:535] client connection to 127.6.57.126:42665 recv error: Network error: failed to read from TLS socket (remote: 127.6.57.126:42665): Connection reset by peer (error 104)
2026-08-06T06:50:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:20 dist-test-slave-5l3k krb5kdc[28823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:20 dist-test-slave-5l3k krb5kdc[28823](info): set up 2 sockets
Aug 06 06:50:20 dist-test-slave-5l3k krb5kdc[28823](info): commencing operation
krb5kdc: starting...
W20260806 06:50:21.020630 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:21 dist-test-slave-5l3k krb5kdc[28823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999021, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:21.135550 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:33063
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:41857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:33063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:21.217828 28837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:21.218024 28837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:21.218047 28837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:21.219213 28837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:21.219256 28837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:21.219271 28837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:21.219285 28837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:21.219296 28837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:21.220659 28837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:41857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:33063
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:21.220920 28837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:21.221145 28837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:21.226042 28843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:21.226176 28837 server_base.cc:1034] running on GCE node
W20260806 06:50:21.226279 28845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:21.226054 28842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:21.226570 28837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:21.226864 28837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:21.228030 28837 hybrid_clock.cc:648] HybridClock initialized: now 1785999021228015 us; error 28 us; skew 500 ppm
Aug 06 06:50:21 dist-test-slave-5l3k krb5kdc[28823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999021, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:21.234957 28837 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:21.235344 28837 webserver.cc:460] Webserver started at http://127.6.57.126:45733/ using document root <none> and password file <none>
I20260806 06:50:21.235565 28837 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:21.235643 28837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:21.235785 28837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:21.236775 28837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "ede7dc2c5d9f4195813d46dbfbcd5444"
format_stamp: "Formatted at 2026-08-06 06:50:21 on dist-test-slave-5l3k"
server_key: "46a9a1a5f5ddf67182093939a45e8326"
server_key_iv: "40e0a9b37b7fb7508eb2626f387aed49"
server_key_version: "encryptionkey@0"
I20260806 06:50:21.237214 28837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "ede7dc2c5d9f4195813d46dbfbcd5444"
format_stamp: "Formatted at 2026-08-06 06:50:21 on dist-test-slave-5l3k"
server_key: "46a9a1a5f5ddf67182093939a45e8326"
server_key_iv: "40e0a9b37b7fb7508eb2626f387aed49"
server_key_version: "encryptionkey@0"
I20260806 06:50:21.238623 28837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:21.239351 28854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:21.239539 28837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:21.239660 28837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "ede7dc2c5d9f4195813d46dbfbcd5444"
format_stamp: "Formatted at 2026-08-06 06:50:21 on dist-test-slave-5l3k"
server_key: "46a9a1a5f5ddf67182093939a45e8326"
server_key_iv: "40e0a9b37b7fb7508eb2626f387aed49"
server_key_version: "encryptionkey@0"
I20260806 06:50:21.239717 28837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:21.250738 28837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:21.252933 28837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:21.253036 28837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:21.257138 28837 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:33063
I20260806 06:50:21.257169 28908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:33063 every 8 connection(s)
I20260806 06:50:21.257501 28837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:21.258440 28909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:21.259935 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28837
I20260806 06:50:21.260020 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:21.260237 6373 external_mini_cluster.cc:1428] Setting key 6c838b8fdff7dc5ba82313138e74a90c
I20260806 06:50:21.261144 28909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: Bootstrap starting.
I20260806 06:50:21.261830 28909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:21.262188 28909 log.cc:824] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:21.262959 28909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: No bootstrap required, opened a new log
I20260806 06:50:21.265199 28909 raft_consensus.cc:348] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } }
I20260806 06:50:21.265345 28909 raft_consensus.cc:374] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:21.265389 28909 raft_consensus.cc:729] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede7dc2c5d9f4195813d46dbfbcd5444, State: Initialized, Role: FOLLOWER
I20260806 06:50:21.265522 28909 consensus_queue.cc:260] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } }
I20260806 06:50:21.265630 28909 raft_consensus.cc:388] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:21.265662 28909 raft_consensus.cc:482] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:21.265702 28909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:21.266299 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:21.260771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39511 (local address 127.6.57.126:33063)
0806 06:50:21.260980 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:21.260989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:21.261022 (+ 33us) server_negotiation.cc:407] Connection header received
0806 06:50:21.261608 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:21.261616 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:21.261674 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:21.261824 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:21.262053 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:21.263400 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:21.264211 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:21.264878 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:21.264998 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:21.265644 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:21.265649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:21.265649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:21.265851 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:21.265887 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:21.265901 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:21.265987 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:21.266077 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 06:50:21.266105 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":95,"threads_started":1}
I20260806 06:50:21.266398 28909 raft_consensus.cc:504] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } }
I20260806 06:50:21.266535 28909 leader_election.cc:302] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ede7dc2c5d9f4195813d46dbfbcd5444; no voters:
I20260806 06:50:21.266692 28909 leader_election.cc:288] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:21.266778 28914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:21.266923 28909 sys_catalog.cc:564] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:21.266991 28914 raft_consensus.cc:686] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [term 1 LEADER]: Becoming Leader. State: Replica: ede7dc2c5d9f4195813d46dbfbcd5444, State: Running, Role: LEADER
I20260806 06:50:21.267139 28914 consensus_queue.cc:237] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } }
I20260806 06:50:21.267313 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:21.267520 28914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ede7dc2c5d9f4195813d46dbfbcd5444. Latest consensus state: current_term: 1 leader_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } } }
I20260806 06:50:21.267760 28914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:21.267685 28915 sys_catalog.cc:455] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ede7dc2c5d9f4195813d46dbfbcd5444" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 33063 } } }
I20260806 06:50:21.268154 28915 sys_catalog.cc:458] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:21.268843 28930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:21.268906 28930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:21.268965 28923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:21.269155 28923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:21.270685 28923 catalog_manager.cc:1347] Generated new cluster ID: f73bf86f83db48388dc2f872ff271ffe
I20260806 06:50:21.270736 28923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:21.278349 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:21.269006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37447 (local address 127.6.57.126:33063)
0806 06:50:21.269059 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:21.269061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:21.274721 (+ 5660us) server_negotiation.cc:407] Connection header received
0806 06:50:21.274795 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:21.274796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:21.274809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:21.274879 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:21.275129 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:21.275940 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:21.277025 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:21.277339 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:21.277412 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:21.278095 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:21.278096 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:21.278096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:21.278100 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:21.278115 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:21.278127 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:21.278188 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:21.278270 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 06:50:21.278280 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 06:50:21.291447 28923 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:21.292151 28923 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:21.294633 28923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ede7dc2c5d9f4195813d46dbfbcd5444: Generated new TSK 0
I20260806 06:50:21.294795 28923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:21.303851 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28837
W20260806 06:50:21.309820 28821 connection.cc:535] client connection to 127.6.57.126:33063 recv error: Network error: failed to read from TLS socket (remote: 127.6.57.126:33063): Connection reset by peer (error 104)
2026-08-06T06:50:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:21 dist-test-slave-5l3k krb5kdc[28942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:21 dist-test-slave-5l3k krb5kdc[28942](info): set up 2 sockets
Aug 06 06:50:21 dist-test-slave-5l3k krb5kdc[28942](info): commencing operation
krb5kdc: starting...
W20260806 06:50:22.352826 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:22 dist-test-slave-5l3k krb5kdc[28942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:22.467352 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43525
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:39675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:22.547904 28956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:22.548086 28956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:22.548110 28956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:22.549247 28956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:22.549289 28956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:22.549304 28956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:22.549316 28956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:22.549371 28956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:22.550858 28956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:39675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:22.551105 28956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:22.551388 28956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:22.556114 28964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:22.556075 28963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:22.556231 28956 server_base.cc:1034] running on GCE node
W20260806 06:50:22.556093 28966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:22.556499 28956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:22.556730 28956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:22.557873 28956 hybrid_clock.cc:648] HybridClock initialized: now 1785999022557852 us; error 36 us; skew 500 ppm
Aug 06 06:50:22 dist-test-slave-5l3k krb5kdc[28942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:22.560380 28956 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:22.560791 28956 webserver.cc:460] Webserver started at http://127.6.57.126:33697/ using document root <none> and password file <none>
I20260806 06:50:22.561002 28956 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:22.561045 28956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:22.561168 28956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:22.562150 28956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "5ed15aff10b24b53a2c8fedf71bf16ba"
format_stamp: "Formatted at 2026-08-06 06:50:22 on dist-test-slave-5l3k"
server_key: "8a6ec8d56103a35760ca181f2dddd945"
server_key_iv: "9922824b70267862a29bda3f37b7d5e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:22.562538 28956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "5ed15aff10b24b53a2c8fedf71bf16ba"
format_stamp: "Formatted at 2026-08-06 06:50:22 on dist-test-slave-5l3k"
server_key: "8a6ec8d56103a35760ca181f2dddd945"
server_key_iv: "9922824b70267862a29bda3f37b7d5e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:22.563802 28956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:22.564515 28973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:22.564687 28956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:22.564790 28956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "5ed15aff10b24b53a2c8fedf71bf16ba"
format_stamp: "Formatted at 2026-08-06 06:50:22 on dist-test-slave-5l3k"
server_key: "8a6ec8d56103a35760ca181f2dddd945"
server_key_iv: "9922824b70267862a29bda3f37b7d5e6"
server_key_version: "encryptionkey@0"
I20260806 06:50:22.564881 28956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:22.568476 28956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:22.570797 28956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:22.570956 28956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:22.575042 28956 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:43525
I20260806 06:50:22.575081 29028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:43525 every 8 connection(s)
I20260806 06:50:22.575417 28956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:22.576510 29029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:22.578706 29029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: Bootstrap starting.
I20260806 06:50:22.579238 29029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:22.579473 29029 log.cc:824] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:22.580120 29029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: No bootstrap required, opened a new log
I20260806 06:50:22.581660 29029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } }
I20260806 06:50:22.581815 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 28956
I20260806 06:50:22.581835 29029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:22.581874 29029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed15aff10b24b53a2c8fedf71bf16ba, State: Initialized, Role: FOLLOWER
I20260806 06:50:22.581920 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:22.582024 29029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } }
I20260806 06:50:22.582129 29029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:22.582149 6373 external_mini_cluster.cc:1428] Setting key a044e2ff4b29897d4ae0323507f7f36f
I20260806 06:50:22.582160 29029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:22.582207 29029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:22.583285 29029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } }
I20260806 06:50:22.583432 29029 leader_election.cc:302] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ed15aff10b24b53a2c8fedf71bf16ba; no voters:
I20260806 06:50:22.583601 29029 leader_election.cc:288] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:22.583652 29034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:22.583827 29034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [term 1 LEADER]: Becoming Leader. State: Replica: 5ed15aff10b24b53a2c8fedf71bf16ba, State: Running, Role: LEADER
I20260806 06:50:22.583854 29029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:22.583956 29034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } }
I20260806 06:50:22.584408 29035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } } }
I20260806 06:50:22.584501 29035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:22.584936 29036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ed15aff10b24b53a2c8fedf71bf16ba. Latest consensus state: current_term: 1 leader_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed15aff10b24b53a2c8fedf71bf16ba" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43525 } } }
I20260806 06:50:22.585064 29036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:22.585383 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:22.582643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51511 (local address 127.6.57.126:43525)
0806 06:50:22.582852 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:22.582860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:22.582884 (+ 24us) server_negotiation.cc:407] Connection header received
0806 06:50:22.582973 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:22.582978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:22.583000 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:22.583102 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:22.584682 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:22.584708 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:22.584709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:22.584714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:22.584764 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:22.584781 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:22.584853 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:22.585149 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 06:50:22.585193 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":114,"threads_started":1}
W20260806 06:50:22.585774 29050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:22.585829 29050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:22.586010 29043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:22.586254 29043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:22.586530 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:22.588557 29043 catalog_manager.cc:1347] Generated new cluster ID: 37019b6c676a4b8eb0cd0473d58d1ff0
I20260806 06:50:22.588624 29043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:22.589571 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:22.588166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42683 (local address 127.6.57.126:43525)
0806 06:50:22.588205 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:22.588207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:22.588654 (+ 447us) server_negotiation.cc:407] Connection header received
0806 06:50:22.588745 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:22.588746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:22.588751 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:22.588798 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:22.589355 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:22.589357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:22.589357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:22.589361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:22.589375 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:22.589379 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:22.589425 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:22.589478 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 06:50:22.589491 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:22.624653 29043 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:22.624902 29043 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:22.630138 29043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ed15aff10b24b53a2c8fedf71bf16ba: Generated new TSK 0
I20260806 06:50:22.630343 29043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:22.632692 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 28956
2026-08-06T06:50:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:22 dist-test-slave-5l3k krb5kdc[29061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:22 dist-test-slave-5l3k krb5kdc[29061](info): set up 2 sockets
Aug 06 06:50:22 dist-test-slave-5l3k krb5kdc[29061](info): commencing operation
krb5kdc: starting...
W20260806 06:50:23.689590 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:23 dist-test-slave-5l3k krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:23.809579 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43925
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:45175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:43925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:23.889698 29077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:23.889930 29077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:23.889967 29077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:23.891072 29077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:23.891137 29077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:23.891162 29077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:23.891189 29077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:23.891219 29077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:23.892599 29077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:45175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:43925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:43925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:23.892890 29077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:23.893114 29077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:23.897835 29082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:23.897836 29085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:23.897862 29083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:23.898154 29077 server_base.cc:1034] running on GCE node
I20260806 06:50:23.898386 29077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:23.898777 29077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:23.899945 29077 hybrid_clock.cc:648] HybridClock initialized: now 1785999023899933 us; error 47 us; skew 500 ppm
Aug 06 06:50:23 dist-test-slave-5l3k krb5kdc[29061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999023, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:23.902321 29077 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:23.902779 29077 webserver.cc:460] Webserver started at http://127.6.57.126:38865/ using document root <none> and password file <none>
I20260806 06:50:23.902971 29077 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:23.903007 29077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:23.903097 29077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:23.904085 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "5cb9347be7b345478205669642953d94"
format_stamp: "Formatted at 2026-08-06 06:50:23 on dist-test-slave-5l3k"
server_key: "c6077dff988846fc10cba56148e9a9a0"
server_key_iv: "90fe1203fc4cfacedab2ad5bce13e105"
server_key_version: "encryptionkey@0"
I20260806 06:50:23.904369 29077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "5cb9347be7b345478205669642953d94"
format_stamp: "Formatted at 2026-08-06 06:50:23 on dist-test-slave-5l3k"
server_key: "c6077dff988846fc10cba56148e9a9a0"
server_key_iv: "90fe1203fc4cfacedab2ad5bce13e105"
server_key_version: "encryptionkey@0"
I20260806 06:50:23.905576 29077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:23.906236 29093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:23.906438 29077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 06:50:23.906554 29077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "5cb9347be7b345478205669642953d94"
format_stamp: "Formatted at 2026-08-06 06:50:23 on dist-test-slave-5l3k"
server_key: "c6077dff988846fc10cba56148e9a9a0"
server_key_iv: "90fe1203fc4cfacedab2ad5bce13e105"
server_key_version: "encryptionkey@0"
I20260806 06:50:23.906634 29077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:23.917088 29077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:23.919399 29077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:23.919538 29077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:23.923462 29077 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:43925
I20260806 06:50:23.923509 29149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:43925 every 8 connection(s)
I20260806 06:50:23.923826 29077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:23.924702 29150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:23.927143 29150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94: Bootstrap starting.
I20260806 06:50:23.927728 29150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:23.928121 29150 log.cc:824] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:23.928846 29150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94: No bootstrap required, opened a new log
I20260806 06:50:23.930419 29150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } }
I20260806 06:50:23.930572 29150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:23.930595 29150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb9347be7b345478205669642953d94, State: Initialized, Role: FOLLOWER
I20260806 06:50:23.930689 29150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } }
I20260806 06:50:23.930755 29150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:23.930776 29150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:23.930802 29150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:23.931591 29150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } }
I20260806 06:50:23.931677 29150 leader_election.cc:302] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cb9347be7b345478205669642953d94; no voters:
I20260806 06:50:23.931802 29150 leader_election.cc:288] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:23.931891 29153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:23.932071 29153 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [term 1 LEADER]: Becoming Leader. State: Replica: 5cb9347be7b345478205669642953d94, State: Running, Role: LEADER
I20260806 06:50:23.932155 29150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:23.932178 29153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } }
I20260806 06:50:23.932627 29154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cb9347be7b345478205669642953d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } } }
I20260806 06:50:23.932622 29155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cb9347be7b345478205669642953d94. Latest consensus state: current_term: 1 leader_uuid: "5cb9347be7b345478205669642953d94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb9347be7b345478205669642953d94" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 43925 } } }
I20260806 06:50:23.932753 29154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:23.932756 29155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:23.933350 29162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:23.933827 29162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:23.934046 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29077
I20260806 06:50:23.934150 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:23.934350 6373 external_mini_cluster.cc:1428] Setting key ec2d57d5b2a26cd63ae18f4b62c3838a
I20260806 06:50:23.935761 29162 catalog_manager.cc:1347] Generated new cluster ID: 3b0fcd242aef43a7b1f32bb01b99b26c
I20260806 06:50:23.935822 29162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 06:50:23 dist-test-slave-5l3k krb5kdc[29061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:23.944733 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:23.934779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34941 (local address 127.6.57.126:43925)
0806 06:50:23.935060 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:23.935068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:23.935250 (+ 182us) server_negotiation.cc:407] Connection header received
0806 06:50:23.935399 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:23.935406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:23.935492 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:23.935635 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:23.936315 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:23.937259 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:23.938037 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:23.938460 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:23.938632 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:23.940947 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:23.940951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:23.940951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:23.940970 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:23.943208 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:23.943588 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:23.943591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:23.943592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:23.943690 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:23.943903 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:23.943904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:23.943905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:23.944129 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:23.944299 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:23.944379 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 06:50:23.944505 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":83,"threads_started":1}
I20260806 06:50:23.945561 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:23.950555 29162 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:23.951145 29162 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:23.953267 29173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:23.946864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39021 (local address 127.6.57.126:43925)
0806 06:50:23.946900 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:23.946902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:23.946996 (+ 94us) server_negotiation.cc:407] Connection header received
0806 06:50:23.947059 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:23.947060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:23.947069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:23.947108 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:23.947618 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:23.948078 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:23.948758 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:23.948990 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:23.949081 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:23.950390 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:23.950391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:23.950392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:23.950398 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:23.952125 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:23.952482 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:23.952483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:23.952484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:23.952553 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:23.952776 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:23.952777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:23.952778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:23.952902 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:23.953001 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:23.953098 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 06:50:23.953184 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:23.956378 29162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cb9347be7b345478205669642953d94: Generated new TSK 0
I20260806 06:50:23.956537 29162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:23.960028 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29077
2026-08-06T06:50:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:24 dist-test-slave-5l3k krb5kdc[29182](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:24 dist-test-slave-5l3k krb5kdc[29182](info): set up 2 sockets
Aug 06 06:50:24 dist-test-slave-5l3k krb5kdc[29182](info): commencing operation
krb5kdc: starting...
W20260806 06:50:25.005100 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:25 dist-test-slave-5l3k krb5kdc[29182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:25.125803 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45233
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:45233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:25.206348 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:25.206610 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:25.206653 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:25.207814 29195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:25.207891 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:25.207914 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:25.207937 29195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:25.207962 29195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:25.209426 29195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:45233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:45233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:25.209705 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:25.209941 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:25.214862 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:25.214885 29201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:25.215093 29195 server_base.cc:1034] running on GCE node
W20260806 06:50:25.215230 29203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:25.215497 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:25.215813 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:25.217020 29195 hybrid_clock.cc:648] HybridClock initialized: now 1785999025216986 us; error 64 us; skew 500 ppm
Aug 06 06:50:25 dist-test-slave-5l3k krb5kdc[29182](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999025, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:25.219996 29195 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:25.220435 29195 webserver.cc:460] Webserver started at http://127.6.57.126:44299/ using document root <none> and password file <none>
I20260806 06:50:25.220662 29195 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:25.220706 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:25.220876 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:25.221997 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "3a1aa4e7355542b0afc37b67d854a3d8"
format_stamp: "Formatted at 2026-08-06 06:50:25 on dist-test-slave-5l3k"
server_key: "d70bd3d27ffd9a1c0a5165d60fb5c35e"
server_key_iv: "e49b0c750451bc1fc23cd796b7efbec9"
server_key_version: "encryptionkey@0"
I20260806 06:50:25.222395 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "3a1aa4e7355542b0afc37b67d854a3d8"
format_stamp: "Formatted at 2026-08-06 06:50:25 on dist-test-slave-5l3k"
server_key: "d70bd3d27ffd9a1c0a5165d60fb5c35e"
server_key_iv: "e49b0c750451bc1fc23cd796b7efbec9"
server_key_version: "encryptionkey@0"
I20260806 06:50:25.223745 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:25.224440 29210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:25.224624 29195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:25.224717 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "3a1aa4e7355542b0afc37b67d854a3d8"
format_stamp: "Formatted at 2026-08-06 06:50:25 on dist-test-slave-5l3k"
server_key: "d70bd3d27ffd9a1c0a5165d60fb5c35e"
server_key_iv: "e49b0c750451bc1fc23cd796b7efbec9"
server_key_version: "encryptionkey@0"
I20260806 06:50:25.224798 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:25.246270 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:25.248533 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:25.248695 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:25.252804 29195 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:45233
I20260806 06:50:25.252866 29267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:45233 every 8 connection(s)
I20260806 06:50:25.253217 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:25.254042 29268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:25.256398 29268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8: Bootstrap starting.
I20260806 06:50:25.256969 29268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:25.257269 29268 log.cc:824] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:25.258015 29268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8: No bootstrap required, opened a new log
I20260806 06:50:25.259631 29268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } }
I20260806 06:50:25.259764 29268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:25.259786 29268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1aa4e7355542b0afc37b67d854a3d8, State: Initialized, Role: FOLLOWER
I20260806 06:50:25.259995 29268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } }
I20260806 06:50:25.260058 29268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:25.260100 29268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:25.260154 29268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:25.260186 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29195
I20260806 06:50:25.260289 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:25.260479 6373 external_mini_cluster.cc:1428] Setting key fd21f9f855d7b036207b4ffc259fe974
I20260806 06:50:25.261312 29268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } }
I20260806 06:50:25.261415 29268 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1aa4e7355542b0afc37b67d854a3d8; no voters:
I20260806 06:50:25.261582 29268 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:25.261693 29273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:25.261860 29273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1aa4e7355542b0afc37b67d854a3d8, State: Running, Role: LEADER
I20260806 06:50:25.261893 29268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:25.262058 29273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } }
I20260806 06:50:25.262593 29275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1aa4e7355542b0afc37b67d854a3d8. Latest consensus state: current_term: 1 leader_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } } }
I20260806 06:50:25.262703 29275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:25.262917 29274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1aa4e7355542b0afc37b67d854a3d8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 45233 } } }
I20260806 06:50:25.263041 29274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:25.263238 29282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:25.263770 29282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:25.265341 29282 catalog_manager.cc:1347] Generated new cluster ID: 604f2adaea024fcbb33c255d6f24fe8e
I20260806 06:50:25.265425 29282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:25.266527 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:25.260986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58493 (local address 127.6.57.126:45233)
0806 06:50:25.261227 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:25.261237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:25.261275 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:50:25.261386 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:25.261392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:25.261457 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:25.261629 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:25.261911 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:25.263348 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:25.264431 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:25.265014 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:25.265159 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:25.265807 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:25.265810 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:25.265811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:25.266034 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:25.266070 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:25.266084 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:25.266153 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:25.266219 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 06:50:25.266277 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260806 06:50:25.267421 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:25.272356 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:25.268960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45133 (local address 127.6.57.126:45233)
0806 06:50:25.268992 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:25.268994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:25.269244 (+ 250us) server_negotiation.cc:407] Connection header received
0806 06:50:25.269271 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:25.269272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:25.269282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:25.269350 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:25.269444 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:25.270156 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:25.271097 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:25.271413 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:25.271491 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:25.272121 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:25.272122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:25.272122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:25.272128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:25.272141 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:25.272145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:25.272205 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:25.272266 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 06:50:25.272288 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:25.276575 29282 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:25.277083 29282 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:25.281699 29282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1aa4e7355542b0afc37b67d854a3d8: Generated new TSK 0
I20260806 06:50:25.281883 29282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:25.285521 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29195
2026-08-06T06:50:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:25 dist-test-slave-5l3k krb5kdc[29302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:25 dist-test-slave-5l3k krb5kdc[29302](info): set up 2 sockets
Aug 06 06:50:25 dist-test-slave-5l3k krb5kdc[29302](info): commencing operation
krb5kdc: starting...
W20260806 06:50:26.328953 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:26 dist-test-slave-5l3k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:26.445434 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41571
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:26.527452 29316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:26.527647 29316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:26.527668 29316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:26.528832 29316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:26.528873 29316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:26.528889 29316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:26.528903 29316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:26.528936 29316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:26.530566 29316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:41571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:26.530825 29316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:26.531070 29316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:26.535892 29322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:26.535966 29324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:26.535918 29321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:26.536053 29316 server_base.cc:1034] running on GCE node
I20260806 06:50:26.536327 29316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:26.536568 29316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:26.537744 29316 hybrid_clock.cc:648] HybridClock initialized: now 1785999026537737 us; error 36 us; skew 500 ppm
Aug 06 06:50:26 dist-test-slave-5l3k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999026, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:26.546329 29316 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:26.546881 29316 webserver.cc:460] Webserver started at http://127.6.57.126:38569/ using document root <none> and password file <none>
I20260806 06:50:26.547132 29316 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:26.547199 29316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:26.547343 29316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:26.548359 29316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "41e2893d95c543f0b4281bc54b964ff8"
format_stamp: "Formatted at 2026-08-06 06:50:26 on dist-test-slave-5l3k"
server_key: "b47bd46f782eef17aae1c1d5fa31a03c"
server_key_iv: "9b6e12bb73dfa785f597cd5881199db8"
server_key_version: "encryptionkey@0"
I20260806 06:50:26.548758 29316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "41e2893d95c543f0b4281bc54b964ff8"
format_stamp: "Formatted at 2026-08-06 06:50:26 on dist-test-slave-5l3k"
server_key: "b47bd46f782eef17aae1c1d5fa31a03c"
server_key_iv: "9b6e12bb73dfa785f597cd5881199db8"
server_key_version: "encryptionkey@0"
I20260806 06:50:26.550195 29316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:26.550995 29331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:26.551178 29316 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:26.551293 29316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "41e2893d95c543f0b4281bc54b964ff8"
format_stamp: "Formatted at 2026-08-06 06:50:26 on dist-test-slave-5l3k"
server_key: "b47bd46f782eef17aae1c1d5fa31a03c"
server_key_iv: "9b6e12bb73dfa785f597cd5881199db8"
server_key_version: "encryptionkey@0"
I20260806 06:50:26.551367 29316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:26.554641 29316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:26.556497 29316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:26.556653 29316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:26.560932 29316 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:41571
I20260806 06:50:26.560966 29384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:41571 every 8 connection(s)
I20260806 06:50:26.561299 29316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:26.562227 29385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:26.564390 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8: Bootstrap starting.
I20260806 06:50:26.564917 29385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:26.565193 29385 log.cc:824] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:26.565862 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8: No bootstrap required, opened a new log
I20260806 06:50:26.567473 29385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } }
I20260806 06:50:26.567613 29385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 06:50:26.567636 29385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e2893d95c543f0b4281bc54b964ff8, State: Initialized, Role: FOLLOWER
I20260806 06:50:26.567729 29385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } }
I20260806 06:50:26.567776 29385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 06:50:26.567795 29385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 06:50:26.567826 29385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:26.568604 29385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } }
I20260806 06:50:26.568691 29385 leader_election.cc:302] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e2893d95c543f0b4281bc54b964ff8; no voters:
I20260806 06:50:26.568807 29385 leader_election.cc:288] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 06:50:26.568872 29389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:26.569092 29389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 41e2893d95c543f0b4281bc54b964ff8, State: Running, Role: LEADER
I20260806 06:50:26.569118 29385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:26.569185 29389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } }
I20260806 06:50:26.569583 29390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41e2893d95c543f0b4281bc54b964ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } } }
I20260806 06:50:26.569684 29390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:26.570057 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29316
I20260806 06:50:26.570199 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:26.570359 6373 external_mini_cluster.cc:1428] Setting key 9e51fe455204c53d80cbebffd01b8a16
I20260806 06:50:26.570505 29391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41e2893d95c543f0b4281bc54b964ff8. Latest consensus state: current_term: 1 leader_uuid: "41e2893d95c543f0b4281bc54b964ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e2893d95c543f0b4281bc54b964ff8" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 41571 } } }
I20260806 06:50:26.570650 29391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:26.570592 29398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:26.571242 29398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:26.572781 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:26.570973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36717 (local address 127.6.57.126:41571)
0806 06:50:26.571221 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:26.571230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:26.571264 (+ 34us) server_negotiation.cc:407] Connection header received
0806 06:50:26.571379 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:26.571386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:26.571415 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:26.571616 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:26.572337 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:26.572341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:26.572341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:26.572347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:26.572395 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:26.572410 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:26.572483 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:26.572558 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 06:50:26.572598 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
I20260806 06:50:26.573194 29398 catalog_manager.cc:1347] Generated new cluster ID: 87738cdc57a945a89a5ec6c31e13f45f
I20260806 06:50:26.573242 29398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:26.574997 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:26.577467 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:26.576574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38893 (local address 127.6.57.126:41571)
0806 06:50:26.576609 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:26.576611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:26.576620 (+ 9us) server_negotiation.cc:407] Connection header received
0806 06:50:26.576662 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:26.576662 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:26.576675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:26.576740 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:26.577259 (+ 519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:26.577261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:26.577261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 06:50:26.577265 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:26.577281 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 06:50:26.577285 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:26.577345 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:26.577397 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:50:26.577409 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:26.604409 29398 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:26.604592 29398 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:26.610808 29398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41e2893d95c543f0b4281bc54b964ff8: Generated new TSK 0
I20260806 06:50:26.611001 29398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:26.621008 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29316
2026-08-06T06:50:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1335 ms)
[----------] 18 tests from AuthTokenIssuingTest (23852 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:26 dist-test-slave-5l3k krb5kdc[29418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:26 dist-test-slave-5l3k krb5kdc[29418](info): set up 2 sockets
Aug 06 06:50:26 dist-test-slave-5l3k krb5kdc[29418](info): commencing operation
krb5kdc: starting...
W20260806 06:50:27.665107 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:27 dist-test-slave-5l3k krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:27.787762 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42599
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:27.869524 29433 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:27.869762 29433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:27.869793 29433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:27.869835 29433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:27.871073 29433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:27.871140 29433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:27.871162 29433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:27.871183 29433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:27.871213 29433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:27.872725 29433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:27.872997 29433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:27.873219 29433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:27.877952 29442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:27.877974 29439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:27.877985 29440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:27.878316 29433 server_base.cc:1034] running on GCE node
I20260806 06:50:27.878557 29433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:27.878794 29433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:27.879959 29433 hybrid_clock.cc:648] HybridClock initialized: now 1785999027879949 us; error 40 us; skew 500 ppm
Aug 06 06:50:27 dist-test-slave-5l3k krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:27.882545 29433 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:27.882923 29433 webserver.cc:460] Webserver started at http://127.6.57.126:38031/ using document root <none> and password file <none>
I20260806 06:50:27.883100 29433 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:27.883138 29433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:27.883224 29433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:27.884174 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "b4930b1e030a41898ef037794fff6f92"
format_stamp: "Formatted at 2026-08-06 06:50:27 on dist-test-slave-5l3k"
server_key: "af45136e3b1bed346b64e6c6dc737b17"
server_key_iv: "da2bd3359b807c62f090eb671ae39b90"
server_key_version: "encryptionkey@0"
I20260806 06:50:27.884457 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "b4930b1e030a41898ef037794fff6f92"
format_stamp: "Formatted at 2026-08-06 06:50:27 on dist-test-slave-5l3k"
server_key: "af45136e3b1bed346b64e6c6dc737b17"
server_key_iv: "da2bd3359b807c62f090eb671ae39b90"
server_key_version: "encryptionkey@0"
I20260806 06:50:27.885689 29433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:27.886379 29449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:27.886539 29433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:27.886636 29433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "b4930b1e030a41898ef037794fff6f92"
format_stamp: "Formatted at 2026-08-06 06:50:27 on dist-test-slave-5l3k"
server_key: "af45136e3b1bed346b64e6c6dc737b17"
server_key_iv: "da2bd3359b807c62f090eb671ae39b90"
server_key_version: "encryptionkey@0"
I20260806 06:50:27.886708 29433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:27.902825 29433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:27.904690 29433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:27.904845 29433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:27.909094 29433 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:42599
I20260806 06:50:27.909178 29503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:42599 every 8 connection(s)
I20260806 06:50:27.909508 29433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:27.910611 29504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:27.912091 29504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:27.912478 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29433
I20260806 06:50:27.912609 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:27.912809 6373 external_mini_cluster.cc:1428] Setting key 856f39441131c71e414eccecf659513d
Aug 06 06:50:27 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.125@KRBTEST.COM; defaulting to no policy
I20260806 06:50:27.922022 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.912860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42599 (local address 127.0.0.1:38218)
0806 06:50:27.913038 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:27.913072 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:27.913568 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:27.913846 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:27.913849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:27.914121 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:27.914274 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:27.914278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:27.915367 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:27.915368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:27.915974 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:27.915976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:27.916463 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:27.916464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:27.916525 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:27.917015 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:27.917022 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:27.918811 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:27.920480 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:27.920488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:27.920490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:27.920770 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:27.920998 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:27.920999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:27.920999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:27.921104 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:27.921515 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:27.921523 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:27.921602 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 06:50:27.921703 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 06:50:27.921822 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260806 06:50:27.922025 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.913079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.6.57.126:42599)
0806 06:50:27.913215 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:27.913219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:27.913250 (+ 31us) server_negotiation.cc:407] Connection header received
0806 06:50:27.913718 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:27.913720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:27.913739 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:27.913842 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:27.914375 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:27.915248 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:27.916070 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:27.916357 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:27.916464 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:27.918899 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:27.918906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:27.918906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:27.918913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:27.920393 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:27.920854 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:27.920856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:27.920857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:27.920929 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:27.921186 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:27.921186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:27.921187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:27.921442 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:27.921503 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:27.921746 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 06:50:27.921879 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260806 06:50:27.922788 29504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:27.923074 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.922994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41172)
0806 06:50:27.923022 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:27.923058 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 06:50:27.923239 29453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
I20260806 06:50:27.923641 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.923561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41184)
0806 06:50:27.923602 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:27.923626 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:50:27.923753 29504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:43663: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
I20260806 06:50:27.939016 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:43663
--webserver_interface=127.6.57.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:50:27.973030 29504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } attempt: 1
I20260806 06:50:27.973548 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.973406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41198)
0806 06:50:27.973473 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:27.973517 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 06:50:27.973946 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:27.973834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41204)
0806 06:50:27.973891 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:27.973919 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 06:50:27.974066 29504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:43663: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
W20260806 06:50:28.021853 29511 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:28.022035 29511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:28.022053 29511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:28.022073 29511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:28.023253 29511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:28.023289 29511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:28.023303 29511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:28.023314 29511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:28.023325 29511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:28.024760 29511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:43663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.57.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:28.024977 29511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:28.025219 29511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:28.029784 29519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:28.029789 29517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:28.029846 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:28.030121 29511 server_base.cc:1034] running on GCE node
I20260806 06:50:28.030372 29511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:28.030709 29511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:28.031880 29511 hybrid_clock.cc:648] HybridClock initialized: now 1785999028031868 us; error 25 us; skew 500 ppm
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:28.034325 29511 init.cc:377] Logged in from keytab as kudu/127.6.57.125@KRBTEST.COM (short username kudu)
I20260806 06:50:28.034787 29511 webserver.cc:460] Webserver started at http://127.6.57.125:42123/ using document root <none> and password file <none>
I20260806 06:50:28.035023 29511 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:28.035089 29511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:28.035243 29511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:28.036252 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data/instance:
uuid: "296d0712e9bd48f99a549ac977812791"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "5424f16da3792ce9e763aaca67183b76"
server_key_iv: "816becd0fa8dc9e60815ab4dd4a397fd"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.036618 29511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal/instance:
uuid: "296d0712e9bd48f99a549ac977812791"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "5424f16da3792ce9e763aaca67183b76"
server_key_iv: "816becd0fa8dc9e60815ab4dd4a397fd"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.037937 29511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 06:50:28.038748 29526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:28.038923 29511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 06:50:28.039029 29511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "296d0712e9bd48f99a549ac977812791"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "5424f16da3792ce9e763aaca67183b76"
server_key_iv: "816becd0fa8dc9e60815ab4dd4a397fd"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.039101 29511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:28.042223 29504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } attempt: 2
I20260806 06:50:28.042626 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.042515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41218)
0806 06:50:28.042565 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.042599 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 06:50:28.042917 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.042860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41234)
0806 06:50:28.042880 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.042902 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 06:50:28.043015 29504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:43663: Network error: Client connection negotiation failed: client connection to 127.6.57.125:43663: connect: Connection refused (error 111)
I20260806 06:50:28.052768 29511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:28.054466 29511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:28.054737 29511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:28.058395 29511 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.125:43663
I20260806 06:50:28.058447 29583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.125:43663 every 8 connection(s)
I20260806 06:50:28.058799 29511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
I20260806 06:50:28.059829 29584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:28.061398 29584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:28.063704 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29511
I20260806 06:50:28.063783 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-1/wal/instance
I20260806 06:50:28.063977 6373 external_mini_cluster.cc:1428] Setting key 7e0edb47895306c3cd4980e04d32115c
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:28.070536 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.062089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.6.57.126:42599)
0806 06:50:28.062292 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.062312 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.062454 (+ 142us) server_negotiation.cc:407] Connection header received
0806 06:50:28.062998 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.063000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.063026 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.063083 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.063680 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.064273 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.065219 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.065442 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.065531 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.067654 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.067655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.067656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.067664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.069104 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.069522 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.069524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.069524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.069590 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.069844 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.069845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.069845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.069960 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.070037 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.070331 (+ 294us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.070412 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180}
I20260806 06:50:28.070679 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.061987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42599 (local address 127.0.0.1:38224)
0806 06:50:28.062291 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.062362 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.062875 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.063153 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.063157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.063377 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.063565 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.063569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.064363 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.064364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.065128 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.065130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.065559 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.065560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.065665 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.066126 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.066134 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.067548 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.069212 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.069220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.069222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.069440 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.069674 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.069675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.069675 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.069779 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.070040 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.070048 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.070182 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.070298 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.070439 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":90,"threads_started":1}
I20260806 06:50:28.071470 29584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.57.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.124@KRBTEST.COM" created.
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:28.078691 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.071727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41250)
0806 06:50:28.071774 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.071784 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.071836 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.072133 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.072134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.072260 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.072303 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.072304 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.073191 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.073193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.073707 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.073709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.074219 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.074220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.074282 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.074752 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.074757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.075837 (+ 1080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.077569 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.077570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.077570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.077738 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.077998 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.077999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.078000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.078054 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.078352 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.078353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.078388 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.078451 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.078566 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:50:28.078756 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.071838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.6.57.125:43663)
0806 06:50:28.071970 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.071974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.071988 (+ 14us) server_negotiation.cc:407] Connection header received
0806 06:50:28.072030 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.072032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.072041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.072116 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.072382 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.073108 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.073809 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.074116 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.074191 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.075919 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.075927 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.075928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.075934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.077459 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.077862 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.077865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.077865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.077925 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.078122 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.078123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.078123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.078279 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.078371 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.078505 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.078619 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260806 06:50:28.079679 29584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:28.080014 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.079899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48130)
0806 06:50:28.079946 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.079992 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 06:50:28.080149 29527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
I20260806 06:50:28.080539 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.080442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48138)
0806 06:50:28.080488 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.080518 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:50:28.080649 29584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:37769: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
I20260806 06:50:28.089480 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:37769
--webserver_interface=127.6.57.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:34491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:50:28.129882 29584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } attempt: 1
I20260806 06:50:28.130473 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.130298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48152)
0806 06:50:28.130395 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.130439 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 06:50:28.131017 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.130886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48160)
0806 06:50:28.130934 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.130965 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:50:28.131142 29584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:37769: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
I20260806 06:50:28.134176 29504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } attempt: 3
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:28.141572 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.134547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41264)
0806 06:50:28.134598 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.134616 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.134682 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.134833 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.134834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.135017 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.135103 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.135105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.135752 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.135754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.136534 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.136537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.136883 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.136884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.136957 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.137402 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.137408 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.138794 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.140485 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.140486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.140487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.140657 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.140864 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.140865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.140866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.140950 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.141212 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.141213 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.141297 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.141370 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.141473 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:50:28.141572 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.134579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.6.57.125:43663)
0806 06:50:28.134621 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.134623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.134661 (+ 38us) server_negotiation.cc:407] Connection header received
0806 06:50:28.134749 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.134750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.134761 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.134812 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.135179 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.135680 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.136617 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.136817 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.136869 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.138892 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.138894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.138894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.138902 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.140368 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.140731 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.140732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.140733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.140793 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.141021 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.141022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.141022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.141131 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.141207 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.141411 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.141485 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 06:50:28.142061 29504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:28.142446 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.142311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48170)
0806 06:50:28.142385 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.142416 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 06:50:28.142795 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.142727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48184)
0806 06:50:28.142759 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.142782 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 06:50:28.142895 29504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:37769: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
W20260806 06:50:28.172012 29591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:28.172245 29591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:28.172279 29591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:28.172318 29591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:28.173465 29591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:28.173520 29591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:28.173542 29591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:28.173568 29591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:28.173597 29591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:28.174044 29504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } attempt: 1
I20260806 06:50:28.174512 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.174343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48186)
0806 06:50:28.174409 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.174445 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:50:28.174907 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.174795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48192)
0806 06:50:28.174838 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.174862 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 06:50:28.175031 29504 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:37769: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
I20260806 06:50:28.175351 29591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:34491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:42599,127.6.57.125:43663,127.6.57.124:37769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:37769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.57.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:28.175635 29591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:28.175885 29591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:28.180749 29596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:28.180764 29599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:28.180810 29597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:28.180940 29591 server_base.cc:1034] running on GCE node
I20260806 06:50:28.181237 29591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:28.181517 29591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:28.182693 29591 hybrid_clock.cc:648] HybridClock initialized: now 1785999028182689 us; error 51 us; skew 500 ppm
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:28.185253 29591 init.cc:377] Logged in from keytab as kudu/127.6.57.124@KRBTEST.COM (short username kudu)
I20260806 06:50:28.185683 29591 webserver.cc:460] Webserver started at http://127.6.57.124:39827/ using document root <none> and password file <none>
I20260806 06:50:28.185900 29591 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:28.185945 29591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:28.186131 29591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:28.187266 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data/instance:
uuid: "3ba11a1b3f4946cb8108eec812e28360"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "33c835ec1ab1481d86688c0841f6cc34"
server_key_iv: "43ecd700fac607a29d0bc7f2eab442fa"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.187654 29591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal/instance:
uuid: "3ba11a1b3f4946cb8108eec812e28360"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "33c835ec1ab1481d86688c0841f6cc34"
server_key_iv: "43ecd700fac607a29d0bc7f2eab442fa"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.188968 29591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 06:50:28.189790 29606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:28.189966 29591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:28.190049 29591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "3ba11a1b3f4946cb8108eec812e28360"
format_stamp: "Formatted at 2026-08-06 06:50:28 on dist-test-slave-5l3k"
server_key: "33c835ec1ab1481d86688c0841f6cc34"
server_key_iv: "43ecd700fac607a29d0bc7f2eab442fa"
server_key_version: "encryptionkey@0"
I20260806 06:50:28.190100 29591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:28.199309 29584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } attempt: 2
I20260806 06:50:28.199800 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.199666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48200)
0806 06:50:28.199728 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.199769 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:50:28.200270 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.200127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48212)
0806 06:50:28.200172 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.200251 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:50:28.200392 29584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:37769: Network error: Client connection negotiation failed: client connection to 127.6.57.124:37769: connect: Connection refused (error 111)
I20260806 06:50:28.208144 29591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:28.209996 29591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:28.210104 29591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:28.214706 29591 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.124:37769
I20260806 06:50:28.215178 29591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
I20260806 06:50:28.215250 29659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.124:37769 every 8 connection(s)
I20260806 06:50:28.216300 29660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:28.217675 29660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:28.224171 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29591
I20260806 06:50:28.224283 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785998763187424-6373-0/minicluster-data/master-2/wal/instance
I20260806 06:50:28.224534 6373 external_mini_cluster.cc:1428] Setting key 19e21fc6309b6237ac42a6226bdce61e
I20260806 06:50:28.226599 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.218339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.6.57.126:42599)
0806 06:50:28.218553 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.218556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.218721 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:50:28.219175 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.219176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.219187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.219229 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.219761 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.220218 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.221159 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.221344 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.221403 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.223441 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.223443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.223443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.223450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.225176 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.225568 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.225570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.225571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.225633 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.225854 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.225855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.225855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.225994 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.226058 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.226378 (+ 320us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.226463 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260806 06:50:28.226698 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.218284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:42599 (local address 127.0.0.1:38226)
0806 06:50:28.218599 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.218645 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.219070 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.219280 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.219282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.219507 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.219695 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.219699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.220303 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.220304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.221076 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.221078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.221445 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.221446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.221538 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.221977 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.221984 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.223362 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.225244 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.225248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.225250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.225499 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.225702 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.225702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.225703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.225793 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.226057 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.226061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.226202 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.226314 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.226454 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":209,"threads_started":1}
I20260806 06:50:28.227288 29660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:28.234608 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.227519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:43663 (local address 127.0.0.1:41278)
0806 06:50:28.227572 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.227581 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.227613 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.227764 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.227765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.227894 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.227955 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.227956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.228993 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.228994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.229487 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.229488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.230035 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.230036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.230084 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.230527 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.230531 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.231817 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.233510 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.233511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.233511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.233712 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.233898 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.233899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.233899 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.233960 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.234239 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.234239 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.234281 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.234354 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.234448 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 06:50:28.234591 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.227610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41278 (local address 127.6.57.125:43663)
0806 06:50:28.227656 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.227658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.227664 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:50:28.227694 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.227695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.227702 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.227762 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.228048 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.228699 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.229575 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.229933 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.230009 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.231904 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.231905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.231906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.231913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.233435 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.233778 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.233779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.233779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.233833 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.234042 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.234043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.234043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.234163 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.234233 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.234393 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.234469 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 06:50:28.234768 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.224926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.6.57.124:37769)
0806 06:50:28.225090 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.225093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.225146 (+ 53us) server_negotiation.cc:407] Connection header received
0806 06:50:28.225226 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.225231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.225245 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.225328 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.226044 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.227327 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.228274 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.228703 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.228776 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.231309 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.231315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.231315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.231321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.233276 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.233673 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.233675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.233675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.233726 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.233978 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.233979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.233979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.234166 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.234294 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.234393 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.234546 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260806 06:50:28.235406 29660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:28.242360 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.235733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48236)
0806 06:50:28.235779 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.235790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.235821 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.235926 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.235927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.236079 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.236118 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.236119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.236925 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.236926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.237661 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.237663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.238040 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.238041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.238077 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.238466 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.238471 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.239641 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.241206 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.241207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.241207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.241380 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.241616 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.241616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.241617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.241674 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.242021 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.242022 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.242063 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.242134 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.242231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260806 06:50:28.242434 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.235802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.6.57.124:37769)
0806 06:50:28.235837 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.235839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.235845 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:50:28.235872 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.235873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.235880 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.235922 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.236187 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.236850 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.237748 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.237965 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.238025 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.239742 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.239743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.239743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.239749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.241102 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.241480 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.241481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.241481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.241536 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.241809 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.241810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.241811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.241951 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.242018 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.242174 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.242291 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":18}
I20260806 06:50:28.243381 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.235591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.6.57.126:42599)
0806 06:50:28.235632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.235635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.235660 (+ 25us) server_negotiation.cc:407] Connection header received
0806 06:50:28.235714 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.235715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.235724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.235782 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.236356 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.237028 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.237933 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.238181 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.238259 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.240297 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.240298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.240299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.240307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.241934 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.242375 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.242376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.242376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.242435 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.242705 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.242706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.242706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.242845 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.242978 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.243219 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.243288 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:50:28.244014 29660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: Bootstrap starting.
I20260806 06:50:28.244963 29660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:28.245262 29660 log.cc:824] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:28.246038 29660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: No bootstrap required, opened a new log
I20260806 06:50:28.247732 29660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.247881 29660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ba11a1b3f4946cb8108eec812e28360, State: Initialized, Role: FOLLOWER
I20260806 06:50:28.248006 29660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.248294 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.248354 29660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:28.248382 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: This master's current role is: FOLLOWER
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:28.250169 29504 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } attempt: 2
W20260806 06:50:28.250270 29680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:28.250325 29680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:28.252444 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.244442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41286 (local address 127.6.57.125:43663)
0806 06:50:28.244480 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.244482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.244527 (+ 45us) server_negotiation.cc:407] Connection header received
0806 06:50:28.244598 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.244599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.244622 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.244678 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.245176 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.245690 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.246392 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.246703 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.246781 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.249048 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.249050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.249050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.249063 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.250888 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.251279 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.251280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.251280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.251334 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.251565 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.251567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.251567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.251697 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.251841 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.252246 (+ 405us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.252325 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 06:50:28.250787 29591 master_runner.cc:186] Error getting master registration for 127.6.57.126:42599: OK, instance_id { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" instance_seqno: 1785999027907966 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:50:28.253229 29591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:50:28.254120 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:50:28.255199 29477 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b4930b1e030a41898ef037794fff6f92"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38240
I20260806 06:50:28.257369 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.250466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.6.57.124:37769)
0806 06:50:28.250614 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.250615 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.250641 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:50:28.250662 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.250663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.250672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.250745 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.251019 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.251533 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.252542 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.252813 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.253101 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.254792 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.254793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.254794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.254801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.256261 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.256605 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.256606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.256607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.256657 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.256858 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.256859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.256859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.256990 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.257068 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.257224 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.257287 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260806 06:50:28.257375 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.250393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48242)
0806 06:50:28.250493 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.250503 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.250548 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.250752 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.250753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.250898 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.250932 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.250933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.251625 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.251627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.252425 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.252428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.252903 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.252904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.252956 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.253378 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.253383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.254707 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.256361 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.256362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.256363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.256534 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.256731 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.256732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.256733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.256788 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.257074 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.257075 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.257116 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.257200 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.257288 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 06:50:28.258761 29504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: Bootstrap starting.
I20260806 06:50:28.259359 29504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:28.259640 29504 log.cc:824] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:28.260608 29504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: No bootstrap required, opened a new log
I20260806 06:50:28.261562 29504 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.261687 29504 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4930b1e030a41898ef037794fff6f92, State: Initialized, Role: FOLLOWER
I20260806 06:50:28.261775 29504 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.262053 29683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.262144 29683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.262163 29477 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:50:28.262208 29477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:28.262864 29477 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.263088 29477 leader_election.cc:288] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [CANDIDATE]: Term 1 election: Requested vote from peers 296d0712e9bd48f99a549ac977812791 (127.6.57.125:43663), 3ba11a1b3f4946cb8108eec812e28360 (127.6.57.124:37769)
I20260806 06:50:28.263387 29633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4930b1e030a41898ef037794fff6f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ba11a1b3f4946cb8108eec812e28360"
I20260806 06:50:28.263553 29633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:28.264348 29633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4930b1e030a41898ef037794fff6f92 in term 1.
I20260806 06:50:28.262099 29504 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:28.264606 29450 leader_election.cc:302] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ba11a1b3f4946cb8108eec812e28360, b4930b1e030a41898ef037794fff6f92; no voters:
I20260806 06:50:28.264755 29683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:28.264899 29683 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [term 1 LEADER]: Becoming Leader. State: Replica: b4930b1e030a41898ef037794fff6f92, State: Running, Role: LEADER
I20260806 06:50:28.264988 29683 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.265460 29683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4930b1e030a41898ef037794fff6f92. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.265547 29683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: This master's current role is: LEADER
W20260806 06:50:28.267140 29701 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:28.267251 29701 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:28.267324 29697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:28.267611 29697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:28.266973 29433 master_runner.cc:186] Error getting master registration for 127.6.57.125:43663: OK, instance_id { permanent_uuid: "296d0712e9bd48f99a549ac977812791" instance_seqno: 1785999028057339 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:50:28.269083 29433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:50:28.275068 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.266518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.6.57.126:42599)
0806 06:50:28.266551 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.266552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.267061 (+ 509us) server_negotiation.cc:407] Connection header received
0806 06:50:28.267160 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.267161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.267172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.267201 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.268027 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.268729 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.269764 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.269960 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.270054 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.271602 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.271604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.271604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.271611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.273310 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.274156 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.274158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.274158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.274214 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.274432 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.274433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.274433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.274592 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.274674 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.274891 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.274967 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:50:28.278966 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.266170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.6.57.124:37769)
0806 06:50:28.266220 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.266222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.270636 (+ 4414us) server_negotiation.cc:407] Connection header received
0806 06:50:28.270721 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.270722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.270731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.270758 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.271391 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.272091 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.273089 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.273390 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.273427 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.275815 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.275816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.275816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.275823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.277607 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.278044 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.278045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.278046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.278103 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.278327 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.278328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.278329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.278453 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.278571 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.278795 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.278870 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:28.280004 29633 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [term 1 FOLLOWER]: Refusing update from remote peer b4930b1e030a41898ef037794fff6f92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:28.280354 29688 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:28.280542 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.266678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41298 (local address 127.6.57.125:43663)
0806 06:50:28.266714 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.266716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.269348 (+ 2632us) server_negotiation.cc:407] Connection header received
0806 06:50:28.269442 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.269444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.269453 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.269481 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.270177 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.270907 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.272520 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.273315 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.273358 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.276839 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.276840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.276841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.276862 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.278704 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.279288 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.279290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.279290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.279364 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.279932 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.279933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.279934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.280077 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.280146 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.280366 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.280443 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:28.281757 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4930b1e030a41898ef037794fff6f92. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.281850 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.291563 29584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } attempt: 3
Aug 06 06:50:28 dist-test-slave-5l3k krb5kdc[29418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:50:28.296188 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.296299 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:28.296382 29706 mvcc.cc:205] Tried to move back new op lower bound from 7315452019832578048 to 7315452019774382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315452019832578048}]
I20260806 06:50:28.296890 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.296968 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.297250 29697 catalog_manager.cc:1347] Generated new cluster ID: 0aef7af032804aa58331c4674d7bfa5e
I20260806 06:50:28.297292 29697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:28.297739 29707 mvcc.cc:205] Tried to move back new op lower bound from 7315452019832578048 to 7315452019774382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315452019832578048}]
I20260806 06:50:28.311283 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.291808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:37769 (local address 127.0.0.1:48254)
0806 06:50:28.291879 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:28.291891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:28.291931 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:28.292072 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:28.292073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:28.292251 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:28.292289 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.292290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.293178 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.293179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.293943 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:28.293944 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.294259 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.294260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:28.294314 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.294752 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:28.294758 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:28.297395 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:28.299831 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.299832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.299833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.300032 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.310682 (+ 10650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:28.310683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:28.310684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:28.310744 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:28.310984 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:28.310985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:28.311034 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 06:50:28.311092 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 06:50:28.311189 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 06:50:28.311568 29664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:28.291911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.6.57.124:37769)
0806 06:50:28.291959 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.291960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.291967 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:28.291996 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.291997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.292007 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.292059 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.292358 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.293092 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.294007 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:28.294190 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:28.294261 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:28.297485 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:28.297487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:28.297487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:28.297494 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:28.299764 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.310530 (+ 10766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.310532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.310532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.310603 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:28.310807 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:28.310807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:28.310808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:28.310932 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:28.310992 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:28.311416 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 06:50:28.311489 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 06:50:28.313033 29584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: Bootstrap starting.
I20260806 06:50:28.313719 29584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:28.313997 29584 log.cc:824] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:28.314858 29584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: No bootstrap required, opened a new log
I20260806 06:50:28.316640 29584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.316761 29584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 296d0712e9bd48f99a549ac977812791, State: Initialized, Role: FOLLOWER
I20260806 06:50:28.316859 29584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } }
I20260806 06:50:28.317174 29584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:28.318364 29555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4930b1e030a41898ef037794fff6f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "296d0712e9bd48f99a549ac977812791"
I20260806 06:50:28.318472 29555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:28.319275 29555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4930b1e030a41898ef037794fff6f92 in term 1.
I20260806 06:50:28.319469 29554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [term 1 FOLLOWER]: Refusing update from remote peer b4930b1e030a41898ef037794fff6f92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:28.319800 29683 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:50:28.322141 29724 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:28.322190 29724 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:28.322270 29725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.322373 29725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.322427 29725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4930b1e030a41898ef037794fff6f92. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.321302 29697 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:28.323616 29697 catalog_manager.cc:1504] Loading token signing keys...
I20260806 06:50:28.322489 29725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.322584 29728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.324851 29728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:28.323997 29726 mvcc.cc:205] Tried to move back new op lower bound from 7315452019832578048 to 7315452019774382080. Current Snapshot: MvccSnapshot[applied={T|T < 7315452019832578048}]
I20260806 06:50:28.325206 29688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4930b1e030a41898ef037794fff6f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4930b1e030a41898ef037794fff6f92" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 42599 } } peers { permanent_uuid: "296d0712e9bd48f99a549ac977812791" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 43663 } } peers { permanent_uuid: "3ba11a1b3f4946cb8108eec812e28360" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 37769 } } }
I20260806 06:50:28.325301 29688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:28.330428 29697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4930b1e030a41898ef037794fff6f92: Generated new TSK 0
I20260806 06:50:28.330787 29697 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 06:50:28.355028 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.353743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41308 (local address 127.6.57.125:43663)
0806 06:50:28.353785 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.353787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.354370 (+ 583us) server_negotiation.cc:407] Connection header received
0806 06:50:28.354470 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.354471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.354499 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.354531 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.354954 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41308: BlockingRecv error: recv got EOF from 127.0.0.1:41308 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:50:28.355722 29507 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.354097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.6.57.126:42599)
0806 06:50:28.354135 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.354137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.355241 (+ 1104us) server_negotiation.cc:407] Connection header received
0806 06:50:28.355329 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.355330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.355339 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.355365 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.355665 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38260: BlockingRecv error: recv got EOF from 127.0.0.1:38260 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:50:28.356657 29664 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.356034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.6.57.124:37769)
0806 06:50:28.356080 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.356081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.356189 (+ 108us) server_negotiation.cc:407] Connection header received
0806 06:50:28.356270 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.356271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.356279 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.356305 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.356595 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48262: BlockingRecv error: recv got EOF from 127.0.0.1:48262 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:50:28.359457 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.358760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.6.57.125:43663)
0806 06:50:28.358793 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.358794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.358959 (+ 165us) server_negotiation.cc:407] Connection header received
0806 06:50:28.359060 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.359060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.359086 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.359117 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.359417 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41314: BlockingRecv error: recv got EOF from 127.0.0.1:41314 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:50:28.362380 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.361569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.6.57.125:43663)
0806 06:50:28.361600 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.361602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.361846 (+ 244us) server_negotiation.cc:407] Connection header received
0806 06:50:28.361993 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.361994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.362003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.362042 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.362340 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41326: BlockingRecv error: recv got EOF from 127.0.0.1:41326 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:50:28.366045 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.365356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.6.57.125:43663)
0806 06:50:28.365399 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.365400 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.365639 (+ 239us) server_negotiation.cc:407] Connection header received
0806 06:50:28.365720 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.365720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.365738 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.365762 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.366010 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41330: BlockingRecv error: recv got EOF from 127.0.0.1:41330 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 06:50:28.371806 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.371018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.6.57.125:43663)
0806 06:50:28.371051 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.371053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.371282 (+ 229us) server_negotiation.cc:407] Connection header received
0806 06:50:28.371375 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.371376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.371386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.371425 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.371757 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41340: BlockingRecv error: recv got EOF from 127.0.0.1:41340 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:50:28.381533 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.380777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.6.57.125:43663)
0806 06:50:28.380818 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.380819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.381051 (+ 232us) server_negotiation.cc:407] Connection header received
0806 06:50:28.381145 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.381146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.381155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.381183 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.381484 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41344: BlockingRecv error: recv got EOF from 127.0.0.1:41344 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:50:28.399577 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.398661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.6.57.125:43663)
0806 06:50:28.398701 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.398703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.398978 (+ 275us) server_negotiation.cc:407] Connection header received
0806 06:50:28.399108 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.399109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.399120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.399154 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.399530 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41358: BlockingRecv error: recv got EOF from 127.0.0.1:41358 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 06:50:28.434038 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.432911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41370 (local address 127.6.57.125:43663)
0806 06:50:28.432971 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.432974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.433301 (+ 327us) server_negotiation.cc:407] Connection header received
0806 06:50:28.433418 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.433419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.433433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.433468 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.433968 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41370: BlockingRecv error: recv got EOF from 127.0.0.1:41370 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 06:50:28.500238 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.499413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.6.57.125:43663)
0806 06:50:28.499523 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.499525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.499538 (+ 13us) server_negotiation.cc:407] Connection header received
0806 06:50:28.499616 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.499617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.499628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.499683 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.500166 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41380: BlockingRecv error: recv got EOF from 127.0.0.1:41380 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 06:50:28.630224 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.629437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.6.57.125:43663)
0806 06:50:28.629525 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.629527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.629629 (+ 102us) server_negotiation.cc:407] Connection header received
0806 06:50:28.629728 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.629730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.629741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.629819 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.630158 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41390: BlockingRecv error: recv got EOF from 127.0.0.1:41390 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 06:50:28.888096 29588 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:28.887398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41404 (local address 127.6.57.125:43663)
0806 06:50:28.887472 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:28.887474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:28.887503 (+ 29us) server_negotiation.cc:407] Connection header received
0806 06:50:28.887567 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:28.887568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:28.887579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:28.887625 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:28.887961 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41404: BlockingRecv error: recv got EOF from 127.0.0.1:41404 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 06:50:29.250754 29680 catalog_manager.cc:1259] Loaded cluster ID: 0aef7af032804aa58331c4674d7bfa5e
I20260806 06:50:29.250821 29680 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: loading cluster ID for follower catalog manager: success
I20260806 06:50:29.251555 29680 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: acquiring CA information for follower catalog manager: success
I20260806 06:50:29.251796 29680 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ba11a1b3f4946cb8108eec812e28360: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:50:29.322664 29724 catalog_manager.cc:1259] Loaded cluster ID: 0aef7af032804aa58331c4674d7bfa5e
I20260806 06:50:29.322748 29724 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: loading cluster ID for follower catalog manager: success
I20260806 06:50:29.323657 29724 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: acquiring CA information for follower catalog manager: success
I20260806 06:50:29.324038 29724 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 296d0712e9bd48f99a549ac977812791: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:50:29.403793 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:29.401221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.6.57.125:43663)
0806 06:50:29.401350 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:29.401352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:29.401363 (+ 11us) server_negotiation.cc:407] Connection header received
0806 06:50:29.401414 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:29.401414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:29.401463 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:29.401546 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:29.401687 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.402172 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.402952 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.403146 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.403217 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:29.403443 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:29.403587 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:29.403675 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:29.403696 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 06:50:29.403718 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260806 06:50:29.408179 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:29.405771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.6.57.124:37769)
0806 06:50:29.405893 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:29.405895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:29.405902 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:29.405935 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:29.405936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:29.405968 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:29.406063 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:29.406164 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.406669 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.407386 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.407589 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.407661 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:29.407832 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:29.407920 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:29.408015 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:29.408067 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 06:50:29.408089 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
I20260806 06:50:29.412704 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:29.409954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.6.57.125:43663)
0806 06:50:29.410012 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:29.410014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:29.410122 (+ 108us) server_negotiation.cc:407] Connection header received
0806 06:50:29.410235 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:29.410236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:29.410247 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:29.410299 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:29.410499 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.411195 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.411964 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.412265 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.412326 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:29.412464 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:29.412495 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:29.412548 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:29.412608 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 06:50:29.412630 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 06:50:29.412942 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:29.409845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.6.57.124:37769)
0806 06:50:29.409893 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:29.409895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:29.410126 (+ 231us) server_negotiation.cc:407] Connection header received
0806 06:50:29.410203 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:29.410204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:29.410213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:29.410261 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:29.410425 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.411156 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.412153 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.412422 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.412477 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:29.412687 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:29.412718 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:29.412747 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:29.412861 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 06:50:29.412880 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 06:50:29.413269 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:29.409841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.6.57.126:42599)
0806 06:50:29.409988 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:29.409990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:29.410106 (+ 116us) server_negotiation.cc:407] Connection header received
0806 06:50:29.410183 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:29.410184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:29.410240 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:29.410338 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:29.410540 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.411007 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.412089 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:29.412413 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:29.412489 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:29.412729 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:29.412838 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:29.412897 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:29.413137 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 06:50:29.413163 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260806 06:50:29.415030 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29433
I20260806 06:50:29.423226 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29511
I20260806 06:50:29.428987 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29591
2026-08-06T06:50:29Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2808 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 06:50:29 dist-test-slave-5l3k krb5kdc[29817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 06:50:29 dist-test-slave-5l3k krb5kdc[29817](info): set up 2 sockets
Aug 06 06:50:29 dist-test-slave-5l3k krb5kdc[29817](info): commencing operation
krb5kdc: starting...
W20260806 06:50:30.464743 6373 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/test-user.keytab.
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T06:50:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T06:50:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.57.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
Entry for principal kudu/127.6.57.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab.
I20260806 06:50:30.583283 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36359
--webserver_interface=127.6.57.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 06:50:30.664181 29832 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:30.664373 29832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:30.664390 29832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:30.664410 29832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:30.665632 29832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:30.665675 29832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:30.665689 29832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:30.665702 29832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:30.665714 29832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:30.667250 29832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.126.keytab
--principal=kudu/127.6.57.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.126:36359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.57.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:30.667533 29832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:30.667784 29832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:30.672690 29841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:30.672855 29832 server_base.cc:1034] running on GCE node
W20260806 06:50:30.672672 29839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:30.672676 29838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:30.673224 29832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:30.673522 29832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:30.674693 29832 hybrid_clock.cc:648] HybridClock initialized: now 1785999030674685 us; error 36 us; skew 500 ppm
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:30.677392 29832 init.cc:377] Logged in from keytab as kudu/127.6.57.126@KRBTEST.COM (short username kudu)
I20260806 06:50:30.677902 29832 webserver.cc:460] Webserver started at http://127.6.57.126:41009/ using document root <none> and password file <none>
I20260806 06:50:30.678124 29832 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:30.678169 29832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:30.678262 29832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:30.679381 29832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data/instance:
uuid: "0d4183d17e11499fa2f5ae12c0d9c371"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "bc7f40289ed55a2915752aa565fcc4bb"
server_key_iv: "d85c4c4f4bc3f43a9d2bff219c348ec3"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.679703 29832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance:
uuid: "0d4183d17e11499fa2f5ae12c0d9c371"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "bc7f40289ed55a2915752aa565fcc4bb"
server_key_iv: "d85c4c4f4bc3f43a9d2bff219c348ec3"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.681051 29832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 06:50:30.681818 29848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:30.682026 29832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 06:50:30.682128 29832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal
uuid: "0d4183d17e11499fa2f5ae12c0d9c371"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "bc7f40289ed55a2915752aa565fcc4bb"
server_key_iv: "d85c4c4f4bc3f43a9d2bff219c348ec3"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.682181 29832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:30.697043 29832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:30.699388 29832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:30.699496 29832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:30.703409 29832 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.126:36359
I20260806 06:50:30.703469 29902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.126:36359 every 8 connection(s)
I20260806 06:50:30.703804 29832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/data/info.pb
I20260806 06:50:30.704823 29903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:30.706385 29903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:30.707950 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29832
I20260806 06:50:30.708086 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-0/wal/instance
I20260806 06:50:30.708320 6373 external_mini_cluster.cc:1428] Setting key 96556a02b4ff70033f5f008f4fd6ee91
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.125@KRBTEST.COM; defaulting to no policy
I20260806 06:50:30.717859 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.707115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36359 (local address 127.0.0.1:49382)
0806 06:50:30.707282 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.707318 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:30.707787 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:30.708197 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:30.708200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:30.708500 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:30.708670 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.708674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.710270 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.710271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.711422 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.711424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.711953 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.711955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.712044 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.712520 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:30.712530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:30.714201 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:30.716203 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.716210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.716212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.716415 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.716715 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.716716 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.716717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.716846 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.717216 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:30.717221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:30.717283 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 06:50:30.717427 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 06:50:30.717679 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":98,"thread_start_us":94,"threads_started":1}
I20260806 06:50:30.717859 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.707137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.6.57.126:36359)
0806 06:50:30.707307 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:30.707316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:30.707405 (+ 89us) server_negotiation.cc:407] Connection header received
0806 06:50:30.708003 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:30.708006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:30.708042 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:30.708156 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:30.708776 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.710150 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.711513 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.711836 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.711932 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.714309 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:30.714317 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:30.714318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:30.714332 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:30.716095 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.716537 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.716540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.716540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.716622 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.716922 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.716923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.716923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.717137 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:30.717236 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:30.717467 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 06:50:30.717610 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260806 06:50:30.718683 29903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:30.719023 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.718898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56260)
0806 06:50:30.718959 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.719005 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 06:50:30.719148 29851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.125@KRBTEST.COM" created.
I20260806 06:50:30.719599 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.719465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56270)
0806 06:50:30.719540 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.719577 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 06:50:30.719708 29903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:46497: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
Entry for principal kudu/127.6.57.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab.
I20260806 06:50:30.733227 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:46497
--webserver_interface=127.6.57.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:50:30.768967 29903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } attempt: 1
I20260806 06:50:30.769551 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.769377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56272)
0806 06:50:30.769461 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.769517 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 06:50:30.770042 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.769927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56280)
0806 06:50:30.769985 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.770021 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 06:50:30.770200 29903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:46497: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
W20260806 06:50:30.815135 29910 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:30.815354 29910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:30.815384 29910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:30.815423 29910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 06:50:30.816511 29910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:30.816571 29910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:30.816596 29910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:30.816632 29910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:30.816658 29910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:30.818216 29910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.125.keytab
--principal=kudu/127.6.57.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.125:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.57.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:30.818531 29910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:30.818792 29910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:30.823562 29915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:30.823571 29918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:30.823589 29916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:30.823854 29910 server_base.cc:1034] running on GCE node
I20260806 06:50:30.824090 29910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:30.824415 29910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:30.825575 29910 hybrid_clock.cc:648] HybridClock initialized: now 1785999030825552 us; error 43 us; skew 500 ppm
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:30.828223 29910 init.cc:377] Logged in from keytab as kudu/127.6.57.125@KRBTEST.COM (short username kudu)
I20260806 06:50:30.828687 29910 webserver.cc:460] Webserver started at http://127.6.57.125:33971/ using document root <none> and password file <none>
I20260806 06:50:30.828930 29910 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:30.829000 29910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:30.829159 29910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:30.830155 29910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data/instance:
uuid: "4794f4d003ac408dab757739b7e34b58"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "df0af3964df41bf09797056231c57a85"
server_key_iv: "b6f9cef6c5972734095dedcde2e72a15"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.830566 29910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal/instance:
uuid: "4794f4d003ac408dab757739b7e34b58"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "df0af3964df41bf09797056231c57a85"
server_key_iv: "b6f9cef6c5972734095dedcde2e72a15"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.831894 29910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 06:50:30.832621 29926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:30.832800 29910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 06:50:30.832908 29910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal
uuid: "4794f4d003ac408dab757739b7e34b58"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "df0af3964df41bf09797056231c57a85"
server_key_iv: "b6f9cef6c5972734095dedcde2e72a15"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.832995 29910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:30.838368 29903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } attempt: 2
I20260806 06:50:30.838825 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.838683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56296)
0806 06:50:30.838759 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.838796 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 06:50:30.839210 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.839110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56306)
0806 06:50:30.839160 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.839191 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:50:30.839300 29903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.125:46497: Network error: Client connection negotiation failed: client connection to 127.6.57.125:46497: connect: Connection refused (error 111)
I20260806 06:50:30.845866 29910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:30.847831 29910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:30.847992 29910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:30.853376 29910 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.125:46497
I20260806 06:50:30.853433 29981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.125:46497 every 8 connection(s)
I20260806 06:50:30.853832 29910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/data/info.pb
I20260806 06:50:30.855141 29982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:30.857547 29982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:30.857872 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29910
I20260806 06:50:30.857977 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-1/wal/instance
I20260806 06:50:30.858217 6373 external_mini_cluster.cc:1428] Setting key f520d9bc67de31dabdbd2f481bef50af
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.57.124@KRBTEST.COM; defaulting to no policy
I20260806 06:50:30.869342 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.859942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49394 (local address 127.6.57.126:36359)
0806 06:50:30.860071 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:30.860074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:30.860086 (+ 12us) server_negotiation.cc:407] Connection header received
0806 06:50:30.860119 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:30.860121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:30.860135 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:30.860212 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:30.860876 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.861458 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.863127 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.863322 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.863388 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.866015 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:30.866017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:30.866018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:30.866027 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:30.867629 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.868163 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.868165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.868166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.868233 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.868537 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.868538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.868539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.868663 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:30.868750 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:30.869145 (+ 395us) server_negotiation.cc:299] Negotiation successful
0806 06:50:30.869215 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.57.124@KRBTEST.COM" created.
I20260806 06:50:30.869547 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.858368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36359 (local address 127.0.0.1:49394)
0806 06:50:30.858615 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.858684 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:30.859168 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:30.860305 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:30.860309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:30.860584 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:30.860774 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.860779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.861568 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.861570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.863025 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.863028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.863451 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.863453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.863633 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.864286 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:30.864298 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:30.865904 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:30.867770 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.867778 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.867780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.868059 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.868311 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.868312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.868313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.868439 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.868769 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:30.868777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:30.868961 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 06:50:30.869090 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 06:50:30.869276 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260806 06:50:30.870196 29982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:30.878680 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.870456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56316)
0806 06:50:30.870691 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.870703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:30.870758 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:30.870903 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:30.870904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:30.871078 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:30.871139 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.871140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.871911 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.871912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.872830 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.872833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.873427 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.873428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.873509 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.873875 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:30.873880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:30.875265 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:30.877218 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.877219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.877219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.877403 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.877649 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.877650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.877650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.877757 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.878240 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:30.878242 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:30.878313 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 06:50:30.878410 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 06:50:30.878525 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260806 06:50:30.878751 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.870556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.6.57.125:46497)
0806 06:50:30.870697 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:30.870701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:30.870751 (+ 50us) server_negotiation.cc:407] Connection header received
0806 06:50:30.870819 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:30.870821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:30.870830 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:30.870885 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:30.871226 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.871843 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.872927 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.873338 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.873403 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.875372 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:30.875382 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:30.875382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:30.875395 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:30.877086 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.877510 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.877513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.877514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.877576 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.877829 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.877830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.877831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.877987 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:30.878258 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:30.878443 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 06:50:30.878600 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260806 06:50:30.879565 29982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:30.879902 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.879793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46356)
0806 06:50:30.879841 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.879881 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 06:50:30.880046 29928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
I20260806 06:50:30.880559 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.880460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46362)
0806 06:50:30.880505 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.880530 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 06:50:30.880697 29982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:38867: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
Entry for principal kudu/127.6.57.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab.
I20260806 06:50:30.885123 6373 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXFWrbB/build/release/bin/kudu
/tmp/dist-test-taskXFWrbB/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:38867
--webserver_interface=127.6.57.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.57.84:32913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 06:50:30.929930 29982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } attempt: 1
I20260806 06:50:30.930394 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.930259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46376)
0806 06:50:30.930327 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.930364 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:50:30.930454 29903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } attempt: 3
I20260806 06:50:30.930827 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.930683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46384)
0806 06:50:30.930752 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.930798 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 06:50:30.930925 29982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:38867: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:30.938076 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.930814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56326)
0806 06:50:30.930871 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.930883 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:30.930935 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:30.931160 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:30.931162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:30.931358 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:30.931445 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.931447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.932146 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.932147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.932993 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:30.932996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.933356 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.933358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:30.933451 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.933910 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:30.933916 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:30.935286 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:30.937090 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.937091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.937091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.937269 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.937491 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:30.937492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:30.937493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:30.937542 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:30.937854 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:30.937854 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:30.937893 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 06:50:30.937942 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 06:50:30.938014 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 06:50:30.938171 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.930880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56326 (local address 127.6.57.125:46497)
0806 06:50:30.931018 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:30.931020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:30.931027 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:30.931062 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:30.931063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:30.931074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:30.931114 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:30.931561 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.932055 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.933080 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:30.933283 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:30.933332 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:30.935419 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:30.935421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:30.935421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:30.935430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:30.937011 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.937342 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.937343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.937344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.937415 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:30.937604 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:30.937604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:30.937605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:30.937760 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:30.937850 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:30.937995 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 06:50:30.938081 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 06:50:30.938575 29903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:30.938922 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.938827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46396)
0806 06:50:30.938861 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.938897 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 06:50:30.939337 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.939237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46408)
0806 06:50:30.939284 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.939310 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 06:50:30.939465 29903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:38867: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
W20260806 06:50:30.969666 29989 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 06:50:30.969844 29989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 06:50:30.969862 29989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 06:50:30.969882 29989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 06:50:30.970641 29903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } attempt: 1
I20260806 06:50:30.971026 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.970901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46414)
0806 06:50:30.970961 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.970999 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 06:50:30.971195 29989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 06:50:30.971259 29989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 06:50:30.971287 29989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 06:50:30.971319 29989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 06:50:30.971347 29989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 06:50:30.971421 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.971327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46430)
0806 06:50:30.971374 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.971398 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 06:50:30.971572 29903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:38867: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
I20260806 06:50:30.972883 29989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.57.84:32913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.57.126:36359,127.6.57.125:46497,127.6.57.124:38867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/krb5kdc/kudu_127.6.57.124.keytab
--principal=kudu/127.6.57.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.57.124:38867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.57.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 05:54:42 UTC on ccycloud.quasar-agbubm.root.comops.site
build id 4569744
I20260806 06:50:30.973186 29989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 06:50:30.973433 29989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 06:50:30.977996 29994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:30.978008 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 06:50:30.978097 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 06:50:30.978364 29989 server_base.cc:1034] running on GCE node
I20260806 06:50:30.978643 29989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 06:50:30.978948 29989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 06:50:30.980114 29989 hybrid_clock.cc:648] HybridClock initialized: now 1785999030980104 us; error 47 us; skew 500 ppm
Aug 06 06:50:30 dist-test-slave-5l3k krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 06:50:30.982573 29989 init.cc:377] Logged in from keytab as kudu/127.6.57.124@KRBTEST.COM (short username kudu)
I20260806 06:50:30.982985 29989 webserver.cc:460] Webserver started at http://127.6.57.124:38445/ using document root <none> and password file <none>
I20260806 06:50:30.983217 29989 fs_manager.cc:359] Metadata directory not provided
I20260806 06:50:30.983284 29989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 06:50:30.983430 29989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 06:50:30.984422 29989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data/instance:
uuid: "95ae5a4236e0466798ca36a5bd6395cf"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "3ab1b05cd6929e85f658020a3104eb10"
server_key_iv: "0855d79823f83847f36286e10bbb4b95"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.984781 29989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal/instance:
uuid: "95ae5a4236e0466798ca36a5bd6395cf"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "3ab1b05cd6929e85f658020a3104eb10"
server_key_iv: "0855d79823f83847f36286e10bbb4b95"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.986107 29989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 06:50:30.986848 30004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:30.987012 29989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 06:50:30.987116 29989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data,/tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal
uuid: "95ae5a4236e0466798ca36a5bd6395cf"
format_stamp: "Formatted at 2026-08-06 06:50:30 on dist-test-slave-5l3k"
server_key: "3ab1b05cd6929e85f658020a3104eb10"
server_key_iv: "0855d79823f83847f36286e10bbb4b95"
server_key_version: "encryptionkey@0"
I20260806 06:50:30.987185 29989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 06:50:30.993741 29989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 06:50:30.995806 29989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 06:50:30.995965 29989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 06:50:30.999068 29982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } attempt: 2
I20260806 06:50:30.999466 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.999331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46432)
0806 06:50:30.999386 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.999439 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 06:50:30.999900 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:30.999758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46434)
0806 06:50:30.999821 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:30.999860 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 06:50:30.999922 29989 rpc_server.cc:307] RPC server started. Bound to: 127.6.57.124:38867
I20260806 06:50:30.999963 30059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.57.124:38867 every 8 connection(s)
W20260806 06:50:31.000006 29982 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.57.124:38867: Network error: Client connection negotiation failed: client connection to 127.6.57.124:38867: connect: Connection refused (error 111)
I20260806 06:50:31.000285 29989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/data/info.pb
I20260806 06:50:31.001235 30060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 06:50:31.002650 30060 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:31.009835 6373 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXFWrbB/build/release/bin/kudu as pid 29989
I20260806 06:50:31.009953 6373 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXFWrbB/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785998763187424-6373-0/minicluster-data/master-2/wal/instance
I20260806 06:50:31.010231 6373 external_mini_cluster.cc:1428] Setting key 109b9a76fcb8b4afdc7228201b2ec13a
I20260806 06:50:31.011296 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.003221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.126:36359 (local address 127.0.0.1:49404)
0806 06:50:31.003365 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:31.003399 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:31.003787 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:31.003981 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:31.003983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:31.004198 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:31.004376 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.004380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.004978 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.004979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.005830 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.005832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.006159 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.006161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.006250 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.006666 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:31.006673 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:31.007910 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:31.009721 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.009729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.009732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.010050 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.010255 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.010256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.010256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.010394 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.010770 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:31.010776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:31.010921 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 06:50:31.011021 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 06:50:31.011143 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260806 06:50:31.011299 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.003237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49404 (local address 127.6.57.126:36359)
0806 06:50:31.003292 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.003293 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.003475 (+ 182us) server_negotiation.cc:407] Connection header received
0806 06:50:31.003876 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.003878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.003887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.003938 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.004442 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.004879 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.005902 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.006070 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.006124 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.008001 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.008002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.008002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.008008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.009641 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.010134 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.010135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.010136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.010187 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.010472 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.010473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.010474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.010660 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.010739 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.011094 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.011177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:50:31.011837 30060 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:50:31.018831 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.012060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.125:46497 (local address 127.0.0.1:56342)
0806 06:50:31.012106 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:31.012115 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:31.012152 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:31.012281 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:31.012282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:31.012383 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:31.012427 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.012428 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.013146 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.013619 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.013620 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.013972 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.014046 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.014593 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:31.014597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:31.015770 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:31.017747 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.017748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.017749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.017951 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.018156 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.018157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.018157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.018222 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.018520 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:31.018521 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:31.018559 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 06:50:31.018651 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 06:50:31.018740 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 06:50:31.018834 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.012138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.6.57.125:46497)
0806 06:50:31.012176 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.012178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.012185 (+ 7us) server_negotiation.cc:407] Connection header received
0806 06:50:31.012208 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.012209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.012217 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.012280 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.012491 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.013052 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013702 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.013905 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013975 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.015855 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.015857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.015857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.015865 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.017652 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.018025 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.018027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.018027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.018087 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.018286 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.018287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.018288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.018398 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.018482 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.018681 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.018752 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:50:31.019130 30060 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } has no permanent_uuid. Determining permanent_uuid...
I20260806 06:50:31.019629 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.010609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.6.57.124:38867)
0806 06:50:31.010790 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.010793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.010829 (+ 36us) server_negotiation.cc:407] Connection header received
0806 06:50:31.010878 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.010880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.010893 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.010960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.011812 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.012759 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013463 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.013772 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.013860 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.016449 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.016457 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.016458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.016464 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.018232 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.018670 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.018673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.018673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.018734 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.018964 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.018965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.018966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.019138 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.019309 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.019416 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.019503 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":121,"threads_started":1}
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.124@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.126@KRBTEST.COM
I20260806 06:50:31.026628 30062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.019360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46450)
0806 06:50:31.019398 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:31.019407 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:31.019437 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:31.019657 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:31.019658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:31.019807 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:31.019837 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.019839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.020820 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.020822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.021333 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.021335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.021740 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.021741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.021793 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.022130 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:31.022135 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:31.023436 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:31.025575 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.025576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.025577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.025770 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.025976 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.025977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.025977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.026043 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.026275 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:31.026276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:31.026311 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 06:50:31.026393 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 06:50:31.026499 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 06:50:31.026877 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.019402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.6.57.124:38867)
0806 06:50:31.019536 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.019538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.019544 (+ 6us) server_negotiation.cc:407] Connection header received
0806 06:50:31.019562 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.019563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.019572 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.019638 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.020061 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.020747 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.021403 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.021671 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.021718 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.023528 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.023529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.023529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.023535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.025485 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.025853 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.025854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.025854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.025911 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.026104 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.026105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.026105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.026201 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.026271 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.026448 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.026569 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260806 06:50:31.027799 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.020380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49416 (local address 127.6.57.126:36359)
0806 06:50:31.020415 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.020417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.020425 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:31.020450 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.020451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.020459 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.020525 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.020980 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.021410 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.022232 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.022423 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.022474 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.024902 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.024904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.024904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.024911 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.026301 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.026972 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.026974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.026974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.027028 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.027272 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.027272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.027272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.027368 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.027531 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.027607 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.027695 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:31.028301 30060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: Bootstrap starting.
I20260806 06:50:31.029064 30060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:31.029372 30060 log.cc:824] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:31.030295 30060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: No bootstrap required, opened a new log
I20260806 06:50:31.032483 30060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.032604 30060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95ae5a4236e0466798ca36a5bd6395cf, State: Initialized, Role: FOLLOWER
I20260806 06:50:31.032683 30060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.57.125@KRBTEST.COM
I20260806 06:50:31.033004 30060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:31.034564 30067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.034723 30067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: This master's current role is: FOLLOWER
W20260806 06:50:31.035300 30079 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:31.035348 30079 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:31.035236 29989 master_runner.cc:186] Error getting master registration for 127.6.57.126:36359: OK, instance_id { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" instance_seqno: 1785999030702187 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:50:31.037027 29989 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:50:31.038166 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.028369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.6.57.125:46497)
0806 06:50:31.028403 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.028405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.028460 (+ 55us) server_negotiation.cc:407] Connection header received
0806 06:50:31.028525 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.028526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.028533 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.028574 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.029160 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.029728 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.030645 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.030813 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.030861 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.033258 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.033260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.033261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.033282 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.035095 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.037390 (+ 2295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.037391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.037391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.037456 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.037644 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.037645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.037645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.037788 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.037949 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.037987 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.038077 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 06:50:31.039732 6373 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 06:50:31.040542 29877 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0d4183d17e11499fa2f5ae12c0d9c371"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49416
I20260806 06:50:31.046698 29903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } attempt: 2
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.126@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:50:31.052999 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.047008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46460 (local address 127.6.57.124:38867)
0806 06:50:31.047058 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.047060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.047070 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:50:31.047104 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.047105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.047122 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.047168 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.047439 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.047859 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.048470 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.048663 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.048752 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.050415 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.050416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.050417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.050424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.051840 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.052191 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.052192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.052193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.052251 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.052457 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.052458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.052458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.052580 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.052658 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.052818 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.052922 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 06:50:31.052992 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.046928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46460)
0806 06:50:31.046988 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:31.047000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:31.047047 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:31.047170 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:31.047170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:31.047308 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:31.047357 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.047358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.047935 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.047936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.048361 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.048362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.048750 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.048751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.048811 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.049215 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:31.049219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:31.050330 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:31.051943 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.051944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.051945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.052112 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.052322 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.052323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.052323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.052392 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.052652 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:31.052653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:31.052694 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 06:50:31.052778 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 06:50:31.052888 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 06:50:31.054131 29903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: Bootstrap starting.
I20260806 06:50:31.054790 29903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:31.055130 29903 log.cc:824] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:31.055824 29903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: No bootstrap required, opened a new log
I20260806 06:50:31.057142 29903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.057267 29903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d4183d17e11499fa2f5ae12c0d9c371, State: Initialized, Role: FOLLOWER
I20260806 06:50:31.057338 29903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.057626 29903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:31.057582 30082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.057679 30082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.057760 29877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 06:50:31.057858 29877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:31.058658 29877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.059139 30032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "95ae5a4236e0466798ca36a5bd6395cf"
I20260806 06:50:31.059234 30032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [term 0 FOLLOWER]: Advancing to term 1
W20260806 06:50:31.060025 30094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:31.060070 30094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:31.060467 30032 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4183d17e11499fa2f5ae12c0d9c371 in term 1.
I20260806 06:50:31.060679 29850 leader_election.cc:302] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d4183d17e11499fa2f5ae12c0d9c371, 95ae5a4236e0466798ca36a5bd6395cf; no voters:
I20260806 06:50:31.060808 30082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 06:50:31.061017 30082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [term 1 LEADER]: Becoming Leader. State: Replica: 0d4183d17e11499fa2f5ae12c0d9c371, State: Running, Role: LEADER
I20260806 06:50:31.061097 30082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.061456 30082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d4183d17e11499fa2f5ae12c0d9c371. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.061535 30082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:31.060793 29877 leader_election.cc:288] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [CANDIDATE]: Term 1 election: Requested vote from peers 4794f4d003ac408dab757739b7e34b58 (127.6.57.125:46497), 95ae5a4236e0466798ca36a5bd6395cf (127.6.57.124:38867)
I20260806 06:50:31.060262 29832 master_runner.cc:186] Error getting master registration for 127.6.57.125:46497: OK, instance_id { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" instance_seqno: 1785999030852000 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 06:50:31.062615 29832 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 06:50:31.062834 30099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 06:50:31.063042 30099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 06:50:31.064656 30032 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [term 1 FOLLOWER]: Refusing update from remote peer 0d4183d17e11499fa2f5ae12c0d9c371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:31.064980 30082 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 06:50:31.066355 30067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d4183d17e11499fa2f5ae12c0d9c371. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.066457 30067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.068073 30104 mvcc.cc:205] Tried to move back new op lower bound from 7315452031238520832 to 7315452031226863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315452031238520832}]
I20260806 06:50:31.068555 30082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.068658 30082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:31.069146 30067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.069242 30067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.070338 30108 mvcc.cc:205] Tried to move back new op lower bound from 7315452031238520832 to 7315452031226863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315452031238520832}]
I20260806 06:50:31.076471 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.063889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.6.57.125:46497)
0806 06:50:31.063928 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.063930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.065354 (+ 1424us) server_negotiation.cc:407] Connection header received
0806 06:50:31.065450 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.065451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.065461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.065502 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.067091 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.067813 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.070640 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.070917 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.070953 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.072504 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.072505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.072506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.072513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.074848 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.075562 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.075563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.075563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.075619 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.075852 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.075853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.075854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.075978 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.076074 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.076281 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.076373 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 06:50:31.077262 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.064721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.6.57.126:36359)
0806 06:50:31.064755 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.064757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.066267 (+ 1510us) server_negotiation.cc:407] Connection header received
0806 06:50:31.066356 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.066356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.066366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.066393 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.068940 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.069600 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.070623 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.070902 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.070972 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.073366 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.073367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.073367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.073374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.074999 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.075801 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.075802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.075802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.075898 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.076256 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.076257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.076257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.076421 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.076503 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.077087 (+ 584us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.077174 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:31.078580 30099 catalog_manager.cc:1347] Generated new cluster ID: e38b17aeafa346b7bd6db8f69456c7d9
I20260806 06:50:31.078637 30099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 06:50:31.082866 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.065013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.6.57.124:38867)
0806 06:50:31.065046 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.065048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.069484 (+ 4436us) server_negotiation.cc:407] Connection header received
0806 06:50:31.069581 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.069582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.069592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.069622 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.070312 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.071034 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.078576 (+ 7542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.078849 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.078890 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.080218 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.080219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.080220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.080226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.081639 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.081979 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.081979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.081980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.082026 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.082179 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.082180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.082180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.082279 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.082365 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.082725 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.082787 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:31.091128 29982 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } attempt: 3
I20260806 06:50:31.091948 30099 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 06:50:31.092659 30099 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 06:50:31 dist-test-slave-5l3k krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785999030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.57.125@KRBTEST.COM for kudu/127.6.57.124@KRBTEST.COM
I20260806 06:50:31.098232 30099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371: Generated new TSK 0
I20260806 06:50:31.098387 30099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 06:50:31.099671 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.091358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.57.124:38867 (local address 127.0.0.1:46488)
0806 06:50:31.091434 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 06:50:31.091444 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 06:50:31.091478 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 06:50:31.091627 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 06:50:31.091628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 06:50:31.091781 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 06:50:31.091823 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.091825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.092803 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.092804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.093721 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 06:50:31.093724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.094230 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.094231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 06:50:31.094301 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.094829 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 06:50:31.094834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 06:50:31.096349 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 06:50:31.098409 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.098410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.098411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.098642 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.098951 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 06:50:31.098952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 06:50:31.098952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 06:50:31.099034 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 06:50:31.099350 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 06:50:31.099351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 06:50:31.099423 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 06:50:31.099485 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 06:50:31.099575 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 06:50:31.100183 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:31.091438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.6.57.124:38867)
0806 06:50:31.091470 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.091472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.091480 (+ 8us) server_negotiation.cc:407] Connection header received
0806 06:50:31.091546 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.091548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.091557 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.091606 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.091928 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.092664 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.093810 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:31.094090 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:31.094164 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:31.096454 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 06:50:31.096455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 06:50:31.096455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 06:50:31.096461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 06:50:31.098253 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.098796 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.098797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.098797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.098867 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 06:50:31.099119 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 06:50:31.099120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 06:50:31.099120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 06:50:31.099251 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 06:50:31.099307 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:31.100010 (+ 703us) server_negotiation.cc:299] Negotiation successful
0806 06:50:31.100086 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 06:50:31.101644 29982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: Bootstrap starting.
I20260806 06:50:31.102450 29982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: Neither blocks nor log segments found. Creating new log.
I20260806 06:50:31.102778 29982 log.cc:824] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: Log is configured to *not* fsync() on all Append() calls
I20260806 06:50:31.107656 29982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: No bootstrap required, opened a new log
I20260806 06:50:31.109678 29982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.109820 29982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4794f4d003ac408dab757739b7e34b58, State: Initialized, Role: FOLLOWER
I20260806 06:50:31.109903 29982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } }
I20260806 06:50:31.110242 29982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 06:50:31.110263 29954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4794f4d003ac408dab757739b7e34b58"
I20260806 06:50:31.110380 29954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [term 0 FOLLOWER]: Advancing to term 1
I20260806 06:50:31.111092 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.111254 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.111768 29954 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d4183d17e11499fa2f5ae12c0d9c371 in term 1.
I20260806 06:50:31.112211 29953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [term 1 FOLLOWER]: Refusing update from remote peer 0d4183d17e11499fa2f5ae12c0d9c371: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 06:50:31.113096 30111 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 06:50:31.113867 30128 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 06:50:31.113919 30128 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 06:50:31.115723 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d4183d17e11499fa2f5ae12c0d9c371. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.115854 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.115938 30115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.116024 30115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 06:50:31.117233 30130 mvcc.cc:205] Tried to move back new op lower bound from 7315452031372877824 to 7315452031226863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315452031238520832}]
W20260806 06:50:31.131835 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.130050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.6.57.125:46497)
0806 06:50:31.130094 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.130096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.131075 (+ 979us) server_negotiation.cc:407] Connection header received
0806 06:50:31.131194 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.131195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.131207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.131244 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.131744 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56360: BlockingRecv error: recv got EOF from 127.0.0.1:56360 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 06:50:31.132725 29906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.130444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.6.57.126:36359)
0806 06:50:31.130510 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.130512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.132220 (+ 1708us) server_negotiation.cc:407] Connection header received
0806 06:50:31.132313 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.132314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.132324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.132353 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.132647 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49440: BlockingRecv error: recv got EOF from 127.0.0.1:49440 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:50:31.133688 30065 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.130785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.6.57.124:38867)
0806 06:50:31.130822 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.130824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.133098 (+ 2274us) server_negotiation.cc:407] Connection header received
0806 06:50:31.133205 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.133207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.133217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.133262 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.133630 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46494: BlockingRecv error: recv got EOF from 127.0.0.1:46494 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:50:31.136350 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.135589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.6.57.125:46497)
0806 06:50:31.135637 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.135639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.135795 (+ 156us) server_negotiation.cc:407] Connection header received
0806 06:50:31.135898 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.135899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.135915 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.135944 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.136309 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56366: BlockingRecv error: recv got EOF from 127.0.0.1:56366 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 06:50:31.139389 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.138632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56380 (local address 127.6.57.125:46497)
0806 06:50:31.138664 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.138666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.138909 (+ 243us) server_negotiation.cc:407] Connection header received
0806 06:50:31.139014 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.139015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.139024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.139051 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.139350 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56380: BlockingRecv error: recv got EOF from 127.0.0.1:56380 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 06:50:31.143281 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.142530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.6.57.125:46497)
0806 06:50:31.142586 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.142588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.142815 (+ 227us) server_negotiation.cc:407] Connection header received
0806 06:50:31.142905 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.142906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.142916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.142944 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.143241 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56382: BlockingRecv error: recv got EOF from 127.0.0.1:56382 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 06:50:31.149124 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.148395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.6.57.125:46497)
0806 06:50:31.148426 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.148428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.148644 (+ 216us) server_negotiation.cc:407] Connection header received
0806 06:50:31.148746 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.148747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.148756 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.148782 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.149072 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56394: BlockingRecv error: recv got EOF from 127.0.0.1:56394 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:50:31.159145 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.158239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56402 (local address 127.6.57.125:46497)
0806 06:50:31.158273 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.158275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.158616 (+ 341us) server_negotiation.cc:407] Connection header received
0806 06:50:31.158716 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.158717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.158727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.158758 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.159085 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56402: BlockingRecv error: recv got EOF from 127.0.0.1:56402 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 06:50:31.177124 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.176304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56418 (local address 127.6.57.125:46497)
0806 06:50:31.176338 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.176339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.176584 (+ 245us) server_negotiation.cc:407] Connection header received
0806 06:50:31.176681 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.176682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.176692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.176720 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.177063 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56418: BlockingRecv error: recv got EOF from 127.0.0.1:56418 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 06:50:31.211529 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.210524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56428 (local address 127.6.57.125:46497)
0806 06:50:31.210567 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.210569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.210858 (+ 289us) server_negotiation.cc:407] Connection header received
0806 06:50:31.210982 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.210984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.210996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.211027 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.211470 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56428: BlockingRecv error: recv got EOF from 127.0.0.1:56428 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 06:50:31.277743 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.277026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56430 (local address 127.6.57.125:46497)
0806 06:50:31.277081 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.277083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.277202 (+ 119us) server_negotiation.cc:407] Connection header received
0806 06:50:31.277241 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.277242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.277254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.277331 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.277667 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56430: BlockingRecv error: recv got EOF from 127.0.0.1:56430 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 06:50:31.407902 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.407035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.6.57.125:46497)
0806 06:50:31.407137 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.407144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.407295 (+ 151us) server_negotiation.cc:407] Connection header received
0806 06:50:31.407355 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.407356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.407372 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.407463 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.407803 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56432: BlockingRecv error: recv got EOF from 127.0.0.1:56432 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 06:50:31.665895 29986 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 06:50:31.665162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.6.57.125:46497)
0806 06:50:31.665239 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:31.665241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:31.665267 (+ 26us) server_negotiation.cc:407] Connection header received
0806 06:50:31.665293 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:31.665294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:31.665305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:31.665349 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 06:50:31.665824 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56438: BlockingRecv error: recv got EOF from 127.0.0.1:56438 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 06:50:31.691715 30198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d4183d17e11499fa2f5ae12c0d9c371" member_type: VOTER last_known_addr { host: "127.6.57.126" port: 36359 } } peers { permanent_uuid: "4794f4d003ac408dab757739b7e34b58" member_type: VOTER last_known_addr { host: "127.6.57.125" port: 46497 } } peers { permanent_uuid: "95ae5a4236e0466798ca36a5bd6395cf" member_type: VOTER last_known_addr { host: "127.6.57.124" port: 38867 } } }
I20260806 06:50:31.691887 30198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d4183d17e11499fa2f5ae12c0d9c371 [sys.catalog]: This master's current role is: LEADER
I20260806 06:50:32.035785 30079 catalog_manager.cc:1259] Loaded cluster ID: e38b17aeafa346b7bd6db8f69456c7d9
I20260806 06:50:32.035878 30079 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: loading cluster ID for follower catalog manager: success
I20260806 06:50:32.036779 30079 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: acquiring CA information for follower catalog manager: success
I20260806 06:50:32.037077 30079 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 95ae5a4236e0466798ca36a5bd6395cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:50:32.114403 30128 catalog_manager.cc:1259] Loaded cluster ID: e38b17aeafa346b7bd6db8f69456c7d9
I20260806 06:50:32.114519 30128 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: loading cluster ID for follower catalog manager: success
I20260806 06:50:32.115276 30128 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: acquiring CA information for follower catalog manager: success
I20260806 06:50:32.115537 30128 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4794f4d003ac408dab757739b7e34b58: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 06:50:32.181460 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:32.179020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.6.57.125:46497)
0806 06:50:32.179152 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:32.179154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:32.179164 (+ 10us) server_negotiation.cc:407] Connection header received
0806 06:50:32.179229 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:32.179230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:32.179281 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:32.179346 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:32.179483 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.179986 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.180714 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.180923 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.180979 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:32.181120 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:32.181238 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:32.181298 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:32.181326 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 06:50:32.181344 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 06:50:32.185662 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:32.183234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.6.57.124:38867)
0806 06:50:32.183357 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:32.183358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:32.183388 (+ 30us) server_negotiation.cc:407] Connection header received
0806 06:50:32.183453 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:32.183454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:32.183488 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:32.183550 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:32.183664 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.184143 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.184954 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.185153 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.185209 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:32.185346 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:32.185433 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:32.185485 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:32.185527 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 06:50:32.185544 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":56,"threads_started":1}
I20260806 06:50:32.189845 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:32.187462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.6.57.124:38867)
0806 06:50:32.187495 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:32.187497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:32.187604 (+ 107us) server_negotiation.cc:407] Connection header received
0806 06:50:32.187650 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:32.187651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:32.187659 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:32.187731 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:32.187869 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.188324 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189057 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.189309 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:32.189581 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:32.189610 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:32.189640 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:32.189761 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 06:50:32.189784 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 06:50:32.190551 30205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:32.187266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56454 (local address 127.6.57.125:46497)
0806 06:50:32.187303 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:32.187305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:32.187474 (+ 169us) server_negotiation.cc:407] Connection header received
0806 06:50:32.187537 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:32.187538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:32.187548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:32.187598 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:32.187770 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.188482 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189643 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.189935 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189997 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:32.190229 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:32.190263 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:32.190345 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:32.190407 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 06:50:32.190443 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 06:50:32.190696 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 06:50:32.187246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.6.57.126:36359)
0806 06:50:32.187364 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 06:50:32.187366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 06:50:32.187477 (+ 111us) server_negotiation.cc:407] Connection header received
0806 06:50:32.187522 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 06:50:32.187523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 06:50:32.187571 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 06:50:32.187642 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 06:50:32.187798 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.188550 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189641 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 06:50:32.189942 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 06:50:32.189992 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 06:50:32.190258 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 06:50:32.190371 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 06:50:32.190446 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 06:50:32.190580 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 06:50:32.190603 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260806 06:50:32.197710 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29832
I20260806 06:50:32.206687 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29910
I20260806 06:50:32.212080 6373 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXFWrbB/build/release/bin/kudu with pid 29989
2026-08-06T06:50:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2783 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5591 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269032 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST